Video Delivery

ABSTRACT

The disclosure relates to a method of delivering a video frame. One implementation may involve spatially partitioning a video frame into a plurality of blocks, encoding at least one of the plurality of blocks of the video frame, and transmitting the at least one of the plurality of blocks of the video frame.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/009,340, filed Sep. 1, 2020, which is a continuation of U.S. patentapplication Ser. No. 15/852,569 filed Dec. 22, 2017, now U.S. Pat. No.10,798,455, the entire disclosures of which are incorporated herein byreference.

BACKGROUND

Advances in data transmission technologies and in digital imaging haveallowed video service providers to provide real-world visual experiencesto users by delivering video content with large frame size and highresolution (or better quality). In comparison to delivery of videocontent having relatively smaller frame size, delivery of video contenthaving increased frame size, e.g., 360 degree video content, may usegreater bandwidth in order to maintain desirable resolution. However, itis expected that the transmission of video content with larger framesize and higher resolution will present new challenges in balancingnetwork bandwidth usage with content quality, and there remains anever-present need for improving the efficiency with which video contentis delivered.

SUMMARY

The following summary is for illustrative purposes only, and is notintended to limit or constrain the detailed description.

Features described herein generally relate to the transmission of video,and to prioritizing transmission resources to emphasize areas of videothat will be of most interest to a user. For example, some immersivevideos have a 360-degree field of available video (e.g., captured usingmultiple cameras pointing in different directions, and/or using speciallenses to capture wide angles of view), and during viewing, the user canlook around, turn his head, move his eyes, rotate/move the camera, etc.to view different areas of the immersive video. In some embodimentsherein, transmission resources may be prioritized to devote moreresources to transmitting the portions of the video that are within theareas being viewed by the user, and fewer resources (or even noresources) to transmitting portions of the video that are outside of theuser's field of view. For example, if the user is viewing a 360-degreevideo and is facing North, then more transmission resources may bedevoted to transmitting the Northern view, and fewer (or no) resourcesmay be devoted to transmitting the Southern view because that is behindthe user's head and outside of the user's field of view.

Even within a particular field of view, the user's eyes may be focusedon a particular area, and further resources may be devoted totransmitting the portion on which the user is focused. For example, theuser may be facing North and seeing a Northern field of view, buthis/her eyes may be focusing on a particular portion within thatNorthern field of view. This area of focus may be given greatertransmission resources. For example, the entire Northern field of viewmay be transmitted at one video resolution via a first stream, and aseparate data stream (e.g., an “enhancement layer”) may be transmittedto carry additional video information pertaining to the area of focus,to enhance that focus area. The enhancement layer may, for example,carry additional video information to support greater pixel density forthe area of focus (e.g., by carrying video data for pixels in betweenthe pixels in the first stream) and/or greater color depth (e.g., bycarrying additional color values to allow the pixels in the area offocus to have a greater range of colors) than for other areas in theNorthern field of view.

A video frame of the 360-degree video described above may be dividedinto, for example, a plurality of graphical blocks or regions, and theblocks may be handled as noted above. For example, the blocks within theNorthern field of view may be transmitted at one resolution, and anenhancement layer may be used to carry additional visual information forthe particular blocks, within that Northern field of view, on which theuser's eyes are focused.

The graphical blocks or regions discussed above may be any portion ofthe video frame. For example, the blocks could be squares resulting fromdividing the frame into a grid. As another example, the blocks could bebased on visual objects in the frame. For example, if pixels in one areaof a frame have a common color, common brightness, are associated with acommon audio level, are moving together, etc., or are otherwiseidentifiable as an object such as a car or a person, then that visualobject may be treated as a block or region as described herein.

The summary here is not an exhaustive listing of the novel featuresdescribed herein, and is not limiting of the claims. These and otherfeatures are described in greater detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentdisclosure will become better understood with regard to the followingdescription, claims, and drawings. The present disclosure is illustratedby way of example, and not limited by, the accompanying figures in whichlike numerals indicate similar elements.

FIG. 1 illustrates an example communication network on which many of thevarious features described herein may be implemented.

FIG. 2 illustrates an exemplary computing device that can be used toimplement any of the methods described herein.

FIG. 3A illustrates an exemplary spatially partitioned video frameaccording to one or more illustrative aspects of the disclosure.

FIG. 3B illustrates an exemplary bitstream representation of apartitioned video frame according to one or more illustrative aspects ofthe disclosure.

FIG. 3C illustrates an exemplary schematic diagram of a communicationsystem which may be implemented to deliver a partitioned video frameaccording to one or more illustrative aspects of the disclosure.

FIG. 4 illustrates an example of fields of view in a video frameaccording to one or more illustrative aspects of the disclosure.

FIG. 5 illustrates an example in which blocks of a partitioned videoframe are packaged as a group according to one or more illustrativeaspects of the disclosure.

FIG. 6 illustrates an example in which a field of view in a video frameis changed for a user in a time period according to one or moreillustrative aspects of the disclosure.

FIG. 7 illustrates an example of delivering groups of blocks when afield of view in a video frame is changed according to one or moreillustrative aspects of the disclosure.

FIG. 8 illustrates an example of encoding each block of a partitionedvideo frame independently from other blocks according to one or moreillustrative aspects of the disclosure.

FIG. 9 illustrates an example of encoding a video frame in which anevent occurs in the video frame according to one or more illustrativeaspects of the disclosure.

FIG. 10 illustrates an example of partial delivery of a partitionedvideo frame according to one or more illustrative aspects of thedisclosure.

FIG. 11 illustrates an example of partial delivery of a partitionedvideo frame with an enhancement layer according to one or moreillustrative aspects of the disclosure.

FIG. 12 is a signaling diagram of a method for video content deliveryaccording to one or more illustrative aspects of the disclosure.

FIG. 13 illustrates an example of statistical data that may be used fordetermining an aggregate area of focus based on statistical dataaccording to one or more illustrative aspects of the disclosure.

FIG. 14 illustrates an example of data that may be used for non-videocontent associated with a video frame according to one or moreillustrative aspects of the disclosure.

FIG. 15 illustrates an example of delivering embedded closed captioningcontent associated with a video frame according to one or moreillustrative aspects of the disclosure.

FIG. 16 illustrates an example of delivering non-embedded closedcaptioning content associated with a video frame according to one ormore illustrative aspects of the disclosure.

FIG. 17 illustrates an example of delivering 3D video content accordingto one or more illustrative aspects of the disclosure.

FIG. 18 is a flow diagram of a method for transmitting video contentaccording to one or more illustrative aspects of the disclosure.

DETAILED DESCRIPTION

In the following description of various illustrative features, referenceis made to the accompanying drawings, which form a part hereof, and inwhich is shown, by way of illustration, various features in whichaspects of the disclosure may be practiced. It is to be understood thatother features may be utilized, and structural and functionalmodifications may be made, without departing from the scope of thepresent disclosure.

FIG. 1 illustrates an example communication network 100 on which many ofthe various features described herein may be implemented. Network 100may be any type of information distribution network, such as satellite,telephone, cellular, wireless, etc. One example may be an optical fibernetwork, a coaxial cable network, or a hybrid fiber/coax distributionnetwork. Such networks 100 use a series of interconnected communicationlinks 101 (e.g., coaxial cables, optical fibers, wireless, etc.) toconnect multiple premises 102 (e.g., businesses, homes, consumerdwellings, etc.) to a local office or headend 103. The local office 103may transmit downstream signals onto the links 101, and each premises102 may have a receiver to receive and process those signals.

There may be one link 101 originating from the local office 103, and itmay be split a number of times to distribute the signals to variouspremises 102 in the vicinity (which may be many miles) of the localoffice 103. The links 101 may include components not illustrated, suchas splitters, filters, amplifiers, etc. to facilitate clear convey ofthe signals, but in general each split introduces a bit of signaldegradation. Some portions of the links 101 may also be implemented withfiber-optic cable, while other portions may be implemented with coaxialcable, other lines, or wireless communication paths.

The local office 103 may include an interface, such as a terminationsystem (TS) 104. More specifically, the termination system 104 may be acable modem termination system (CMTS), which may be a computing deviceconfigured to manage communications between devices on the network oflinks 101 and backend devices such as servers 105-107 (to be discussedfurther below). The termination system 104 may be as specified in astandard, such as the Data Over Cable Service Interface Specification(DOCSIS) standard, published by Cable Television Laboratories, Inc.(a.k.a. CableLabs), or it may be a similar or modified device instead.The termination system 104 may be configured to place data on one ormore downstream frequencies to be received by modems at the variouspremises 102, and to receive upstream signals from those modems on oneor more upstream frequencies.

The local office 103 may also include one or more network interfaces108, which can permit the local office 103 to communicate with variousother external networks 109. These networks 109 may include, forexample, networks of Internet devices, telephone networks, cellulartelephone networks, fiber optic networks, local wireless networks (e.g.,WiMAX), satellite networks, and any other desired network, and thenetwork interface 108 may include the corresponding circuitry needed tocommunicate on the external networks 109, and to other devices on thenetwork such as a cellular telephone network and its corresponding cellphones.

As noted above, the local office 103 may include a variety of servers105-107 that may be configured to perform various functions. Forexample, the local office 103 may include a push notification server105. The push notification server 105 may generate push notifications todeliver data and/or commands to the various premises 102 in the network(or more specifically, to the devices in the premises 102 that areconfigured to detect such notifications). The local office 103 may alsoinclude a content server 106. The content server 106 may be one or morecomputing devices that are configured to provide content to users attheir premises. This content may be, for example, video on demandmovies, 360 degree video streams, television programs, songs, textlistings, etc. The content server 106 may include software to validateuser identities and entitlements, to locate and retrieve requestedcontent, to encrypt the content, and to initiate delivery (e.g.,streaming) of the content to the requesting user(s) and/or device(s).

Video content generally refers to information displayed (or displayable)to a user in the form of one or more images. Video content may haveassociated audio content, e.g., information presented (or presentable)to a user in audio form. Video content may have associated closedcaptioning content. Video content can be communicated or stored in theform of data. Video content data providing a digital representation of aparticular video content can be created using any of various encodingtechniques. Such encoding techniques include, but are not limited to,compression according to a Motion Picture Experts Group (MPEG) standard(e.g., MPEG-2), compression according to the ITU-T H.264 (ISO/IECMPEG-4) advanced video coding (AVC) standard, etc. Video content datamay be included in a single data stream that also includes associatedaudio content, or associated audio content may be carried in a streamseparated from an audio stream. A video content data stream can betransmitted via the communication network 100, by modulating some typeof signal (e.g., an optical signal, an RF carrier signal, an electricalsignal carried over a twisted pair) that is communicated over some typeof medium (e.g., optical fiber, coaxial cable, twisted pair conductor,free space, etc.) using one or more of various types of communicationprotocols (e.g., internet protocol). The received signal may then beprocessed by the premise 102 a to extract the video content data streamand be displayed by the display device 112 of the premise 102 a. Inaddition to demodulating the received signal, such extraction mayinclude demultiplexing by isolating a signal carried on a particularoptical wavelength or RF frequency from signals on other wavelengths orfrequencies, by isolating certain data segments from other datasegments, and/or by other types of demultiplexing techniques. Once thedata stream has been extracted, data from that stream can then bedecoded and used to generate appropriate electrical signals. Thoseelectrical signals can then be output to a display device, for example,by the aforementioned display device 112, so as to cause the displaydevice to present the video content on a display screen. Video contentdata can also be stored in some type of storage device (e.g., a magneticdisk drive) and then later retrieved for decoding and presentation in asimilar manner.

Video content may be composed of more than one video frames eachincluding a raster of pixels. During delivery of the video content, eachvideo frame is delivered to a user device in a predetermined order. Insome examples, each video frame may be spatially partitioned and encodedbefore being transmitted to a user device or multiple user devices. Theuser device may be implemented with any of numerous types of devices,including but not limited to, display devices 112 (e.g., VR (virtualreality) headset, television, high definition television (HDTV), hostviewing device, monitor, game playing device, etc.), additional STBs orDVRs 113, personal computers 114, laptop computers 115, wireless devices116 (e.g., wireless routers, wireless laptops, notebooks, tablets andnetbooks, cordless phones (e.g., Digital Enhanced CordlessTelephone—DECT phones), mobile phones, mobile televisions, personaldigital assistants (PDA), etc.), landline phones 117 (e.g. Voice overInternet Protocol—VoIP phones), and any other desired devices. It shouldbe understood that the user device may be any type of electronic devicethat may display the video content. In reference to FIG. 1, the localoffice 103 may further include one or more application servers 107. Anapplication server 107 may be a computing device configured to offer anydesired service, and may run various languages and operating systems(e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD, Ubuntu,Redhat, HTML5, JavaScript, AJAX, and COMET). For example, an applicationserver may be responsible for collecting television program listingsinformation and generating a data download for electronic program guidelistings. In another example, the application server may be responsiblefor monitoring user viewing habits or for determining a user's area offocus during viewing of video content. In another example, theapplication server may spatially partition a video frame into aplurality of blocks, encode at least one of the blocks, and/or transmitall or a portion of the video frame to the premises 102. In anotherexample, all or a portion of any one of methods of the presentdisclosure may be implemented on any of the push notification server105, the content server 106, the application server 107, and othervarious servers or components, or on any combination of these servers.Although shown separately, one of ordinary skill in the art willappreciate that the push notification server 105, the content server106, and the application server 107 may be combined. Further, here thepush notification server 105, the content server 106, and theapplication server 107 are shown generally, and it will be understoodthat they may each contain memory storing computer executableinstructions to cause a processor thereof to perform steps describedherein and/or a storage for storing data. An exemplary premise 102 a,such as a home, may include an interface 120. The interface 120 caninclude any communication circuitry needed to allow a device tocommunicate on one or more links 101 with other devices in the network100. For example, the interface 120 may include a modem 110, which mayinclude transmitters and receivers used to communicate on the links 101and with the local office 103. The modem 110 may be, for example, acoaxial cable modem (for coaxial cable lines 101), a fiber interfacenode (for fiber optic lines 101), twisted-pair telephone modem, cellulartelephone transceiver, satellite transceiver, local wi-fi router oraccess point, or any other desired modem device. Also, although only onemodem is shown in FIG. 1, a plurality of modems operating in parallelmay be implemented within the interface 120. Further, the interface 120may include a gateway 111. The modem 110 may be connected to, or be apart of, the gateway 111. The gateway 111 may be a computing device thatcommunicates with the modem(s) 110 to allow one or more other devices inthe premises 102 a, to communicate with the local office 103 and otherdevices beyond the local office 103. The gateway 111 may be a set-topbox (STB), digital video recorder (DVR), computer server, or any otherdesired computing device. The gateway 111 may also include (not shown)local network interfaces to provide communication signals to requestinguser entities/devices in the premises 102 a, such as display devices112, additional STBs or DVRs 113, personal computers 114, laptopcomputers 115, wireless devices 116, landline phones 117, and any otherdesired devices. Examples of the local network interfaces includeMultimedia Over Coax Alliance (MoCA) interfaces, Ethernet interfaces,universal serial bus (USB) interfaces, wireless interfaces (e.g., IEEE802.11, IEEE 802.15), analog twisted pair interfaces, Bluetoothinterfaces, and others.

FIG. 2 illustrates an exemplary computing device that can be used toimplement any of the methods described herein. It should be understoodthat servers, user devices, and other related components of thecommunication network mentioned in this disclosure may be computingdevices implemented with all or a portion of the hardware elements ofFIG. 2.

The computing device 200 may include one or more processors 201, whichmay execute instructions of a computer program to perform any of thefeatures described herein. The instructions may be stored in any type ofnon-transitory computer-readable medium or memory, to configure theoperation of the processor 201. For example, instructions may be storedin a read-only memory (ROM) 202, a random access memory (RAM) 203,removable media 204, such as a Universal Serial Bus (USB) drive, compactdisk (CD) or digital versatile disk (DVD), floppy disk drive, or anyother desired storage medium. Instructions may also be stored in anattached (or internal) hard drive 205. The computing device 200 mayinclude or be coupled to one or more output devices, such as a display206 (e.g., an external television), and may include one or more outputdevice controllers 207, such as a video processor. There may also be oneor more user input devices 208, such as a remote control, keyboard,mouse, touch screen, microphone, etc. The computing device 200 may alsoinclude one or more network interfaces, such as a network input/output(I/O) circuit 209 (e.g., a network card) to communicate with an externalnetwork 210. The network input/output circuit 209 may be a wiredinterface, wireless interface, or a combination of the two. In someembodiments, the network input/output circuit 209 may include a modem(e.g., a cable modem), and the external network 210 may include thecommunication links 101 discussed above, the external network 109, anin-home network, a provider's wireless, coaxial, fiber, or hybridfiber/coaxial distribution system (e.g., a DOCSIS network), or any otherdesired network. Additionally, the device may include alocation-detecting device, such as a global positioning system (GPS)microprocessor 211, which can be configured to receive and processglobal positioning signals and determine, with possible assistance froman external server and antenna, a geographic position of the device.

The example of FIG. 2 is a hardware configuration, although theillustrated components may be implemented as software as well.Modifications may be made to add, remove, combine, divide, etc.components of the computing device 200 as desired. Additionally, thecomponents illustrated may be implemented using basic computing devicesand components, and the same components (e.g., processor 201, ROMstorage 202, display 206, etc.) may be used to implement any of theother computing devices and components described herein. For example,the various components herein may be implemented using computing deviceshaving components such as a processor executing computer-executableinstructions stored on a computer-readable medium, as illustrated inFIG. 2. Some or all of the entities described herein may be softwarebased, and may co-exist in a common physical platform (e.g., arequesting entity can be a separate software process and program from adependent entity, both of which may be executed as software on a commoncomputing device).

One or more aspects of the disclosure may be embodied in acomputer-usable data and/or computer-executable instructions, such as inone or more program modules, executed by one or more computers or otherdevices. Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types when executed by a processor ina computer or other data processing device. The computer executableinstructions may be stored on one or more non-transitory computerreadable media such as a hard disk, optical disk, removable storagemedia, solid state memory, ROM, etc. As will be appreciated by one ofskill in the art, the functionality of the program modules may becombined or distributed as desired in various embodiments. In addition,the functionality may be embodied in whole or in part in firmware orhardware equivalents such as integrated circuits, field programmablegate arrays (FPGA), and the like. Particular data structures may be usedto more effectively implement one or more aspects of the disclosure, andsuch data structures are contemplated within the scope of computerexecutable instructions and computer-usable data described herein.

One of ordinary skill in the art will appreciate that some componentsillustrated in FIG. 2 may be omitted in the computing device 200according to design particulars. For example, when the computing device200 is configured as a server, the GPS 211 may be omitted.

As mentioned above, some video content may offer more images beyond whata user is able to see, e.g., a field of view of a user. For example, in360-degree video, the user may manipulate a camera (or turn his/herhead, if using a virtual reality headset) to look at different areas ina 360-degree image. The 360-degree video may offer video at 60 framesper second, and delivery of such video content having such a frame sizelarger than a field of view of a user may require more efficientbandwidth allocation in order to balance network bandwidth usage withcontent quality. Hereinafter, delivery of video content having a framesize larger than a field of view of a user according to variousembodiments will be described. The present disclosure, however, is notlimited thereto. For example, video content having a frame size smallerthan 360 degree video content can also be delivered according variousembodiments of the present disclosure. The smaller size frames mayinclude, for example, hemisphere video frames or quadrant video frames.

In one implementation, during creation of 360 degree video content, alldirections of a view may be simultaneously recorded with, for example,omnidirectional cameras. An omnidirectional camera may cover a360-degree horizontal plane or approximately an entire sphere. Duringplayback of the 360 degree video content, a user may select whichportion of the 360 video content the user wishes to view. A 360 degreevideo frame may be larger in size than a narrower angle video frame,because the 360 degree frame may comprise more information to bedisplayed in comparison with the narrower angle video frame. Thus,transmission of a 360 degree video frame may use more bandwidth thantransmission of a narrower angle video frame.

To save bandwidth, a portion of, rather than an entirety of, the 360degree video frame may be delivered to the user in the communicationnetwork, by, for example, the push notification server 105 and/or thecontent server 106 and/or application server 107 or other variousservers or components described above with reference to FIG. 1. Thepresent disclosure, however, is not limited thereto. In oneimplementation, a 360 degree video frame may be delivered to a userdevice during a predetermined period or when predetermined criteria aresatisfied. These features will be further described with reference toFIGS. 3A through 18, described below.

FIG. 3A illustrates an exemplary spatially-partitioned video frame 30according to one or more illustrative aspects of the disclosure. Theframe 30 shown in FIG. 3A is shown as a rectangle for ease of reference,but in a 360-degree video frame the actual image frame may appear as asphere. FIG. 3A shows an exemplary two-dimensional (2-D) video frame,and an exemplary three-dimensional (3-D) video frame will be shown inFIG. 17 (which will be discussed later). The frame 30 may be dividedinto smaller portions, illustrated as blocks in FIG. 3A, for ease ofprocessing as described herein. In the FIG. 3A example, the blocks maybe identified by coordinates in the horizontal and vertical indicesshown in the figure (e.g., B_(ij)).

In one example embodiment, the 360 degree video frame 30 may bepartially, rather than entirely, delivered to one or more user devices.This may conserve transmission and processing resources. As illustratedin FIG. 3A, the video frame 30 may be spatially partitioned intomultiple blocks, for example the individual blocks B_(i,j) shown in theFIG. 3A grid, by a server (as will be described later). Blocks of thevideo frame 30 may be encoded multiple times, each for a differenttransmission bitrate, and accordingly the blocks may comprisemulti-bitrate (MBR) blocks. The higher bitrate versions may carry higherresolution versions of the block of the image. Higher resolutionversions of blocks provide better quality for the block of the image. Byhaving multiple versions of each block available, the server may choosea higher bitrate version of some blocks, and a lower bitrate version ofother blocks, depending on, for example, what portion(s) of the videoframe 30 the user's eyes are looking at or focusing on. Althoughdescribed as being partitioned into MBR blocks, it should be understoodthat the video frame 30 may be partitioned into other suitable types ofvideo blocks or combinations of types of video blocks. Also, the blocksin FIG. 3A are shown as quadrilaterals/squares, but in implementationthe blocks may take on different shapes.

As mentioned above, the video frame 30 may be partitioned into aplurality of blocks B_(i,j) each of which may contain one or morepixels. In some examples, the shape of the pixels may be square, andthus the shape of the block may be square. In some other examples, theshape of the pixels might not be square, and thus the shape of theblocks might not be square and might have curved edges. In still someother examples, the blocks may have different sizes for different areasof the video frame, and such examples will be further discussed later.In the example shown in FIG. 3A, the video frame 30 is evenlypartitioned into 20×8 square blocks and each block B_(i,j) can beidentified by its corresponding horizontal index i and vertical index j.It should be understood that partitioning of a video frame can vary fordifferent video frames and should not be limited to the particularexample shown in FIG. 3A. For example, the video frame 30 may bespatially partitioned into different number of blocks and each block mayhave a shape the same as or different from each other. In some examples,the blocks may be in any polygonal shapes, e.g., rectangular ortriangle, other than the square shape shown in FIG. 3A, or curvedshapes, e.g., circular, oval, or in any other non-polygonal shapes. Ashape of one block may be the same as or different from that of anotherblock in the same video frame, and an area of one block may be the sameas or different from that of another block in the same video frame. Inone example, some regions may be encoded using smaller blocks than otherregions, so that more data resources may be allocated to encode the someregions than the other regions.

FIG. 3B illustrates an exemplary bitstream of a partitioned video frameaccording to one or more illustrative aspects of the disclosure. Forexample, FIG. 3B may be a bitstream representation of the partitionedvideo frame of FIG. 3A. When blocks B_(i,j) of the video frame 30 shownin FIG. 3A are encoded into a bitstream having MPEG-4 format, a header330 may comprise information representing an address of each encodedblock B_(i,j) in images of the video frame, a size of each encodedblocks B_(i,j), one or more designated bitrates for each encoded blockB_(i,j), and other parameters describing each encoded block B_(i,j). Apayload 340 of the bitstream may include data of all or a portion of theencoded blocks. In one implementation, one or more blocks that aredetermined to not to be viewed or requested might not be encoded orincluded within the bitstream. In this implementation, the payload 340may comprise fewer blocks than if all blocks were transmitted. It shouldalso be understood that the sequence of the encoded blocks in thepayload 340 can be different from B_(1,1), B_(2,1), . . . B_(i,j), . . .B_(20,8) as illustrated in FIG. 3B. It should also be understood thatencoding the blocks B_(i,j) into MPEG-4 is an example for explanationand the present disclosure should not be limited thereto. The blocksB_(i,j) may be encoded into one or more other compressed formats.

Referring back to FIG. 3A, in one implementation, a portion of the videoframe 30, such as a field of view 305 (as will be described later) ofthe video frame 30 may be selected by the server to be transmitted tothe user device. In this implementation, the portion of the video frame30 outside of the field of view 305 might not be transmitted to the userdevice. In the illustration of FIG. 3A, the blocks in the matrix

$\begin{matrix}B_{6,7} & \cdots & B_{12,7} \\ \vdots & \ddots & \vdots \\B_{6,3} & \cdots & B_{12,3}\end{matrix}$

are disposed within or overlap the boundary of the field of view 305,and thus may be associated with the field of view 305.

In one implementation, the server may transmit the encoded blocksassociated with the field of view 305 to the user device, and nottransmit other blocks to the user device. The blocks associated with thefield of view 305 may comprise blocks completely within or partiallywithin the field of view 305. In one example, the blocks associated withthe field of view 305 may comprise blocks having boundaries that overlapthe field of view 305. The user may only download and view the portiondisposed within the field of view 305 at a given moment. Therefore,delivering the blocks associated with the field of view 305, withoutdelivering the entire video frame to the user device, might not impairthe view effect.

In one example, during a predetermined period or when predeterminedcriteria are satisfied, the server may transmit all of, rather than aportion of, the blocks to the user device or user devices. Thesefeatures will be further described below with reference to FIG. 12.

The field of view 305 may represent the portion of the overall imagethat the user can see at a given moment in time. The field of view 305may be dependent on one or more of the user's viewing angle, the degreeof zoom, or the size of a screen of the user's display device (which maybe determined by the manufacturer of the display device). For a certaintype of display device, when the user's eyes are geometrically fixedrelative to the display device at a given moment, the field of view maybe determined at the given moment. For example, when a user wears a VRheadset in a way that the VR headset does not move relative to theuser's eyes, the field of view may be determined at a given moment basedon the direction the user is looking. In some cases, the user maymanually select or change the field of view by selecting areas that theuser wishes to view, such as by entering navigational commands on acontroller. If viewing on an interactive display, the user could tap ona different portion of the field of view to cause the field of view tore-center on the tapped point. In some cases, the user may voluntarilychoose a smaller field of view 305 in order to minimize the amount ofdata that needs to be transmitted, so that the user may have fewerinterruptions due to network delays. In some cases, the field of view305 may be a field of view that is displayed to the user at a givenmoment, and the exact area may be determined by the user device by usingeye tracking technology and data representing the exact area may be sentthrough upstream signals to switches and routers between the server andthe user device, or directly to the server. The aforementionedtechniques to determine the field of view can be combined or modified inconsideration of user comfort or other design particulars.

In the cases where the data representing the area of the field of viewis sent through upstream signals and is cached at the switches androuters, each switch/router may learn and determine the sum of the fieldof views of the users of the user devices that switch/router is serving.In such cases, the server may broadcast the entire video frame. When aswitch/router determines that all of the user devices that theswitch/router is serving are not viewing a block (e.g., block B_(1,1)),the switch/router can modify the data stream that is broadcasted fromthe server by dropping block B_(1,1) from the data stream and send themodified data stream to the user devices that the switch/router isserving.

It should be understood that even though the server is used as anexample in describing delivery of video frames to the user devices inthe present disclosure, in some implementations, the switches androuters may perform some of the operations discussed in the presentdisclosure to realize the delivery of video frames.

In some examples, the field of view might not be dictated by the user,and instead could be determined by a source of the video, such that theuser might not be able to see anything other than the field of viewchosen by the source of the video. In this case, the field of view 305may be selected and determined by a video editor. For example, in avideo game where a user's character may look through a sight scope of arifle, the video editor may black out the peripheral area of the videoframe and confine the field of view within the central region of thevideo frame, thereby imitating the scope of the rifle. The video editormay be one or more people who produce or edit or reassemble the videocontent, or may be a computing device (which may be implemented with thecomputing device as shown in FIG. 2) configured to produce or edit orreassemble the video content. The video editor may a video editor, anofficer operating the server, or personnel independent from the server.When the video editor is a computing device, the video editor may be apart of or an entirety of the server or a computing device independentof the server. In some cases, the video editor may black out certainareas of a video frame by tagging blocks comprising these certain areasas not to be transmitted and may confine the field of view 305 within acertain region of the video frame. In one implementation, if the fieldof view is determined by the video editor, metadata may be used to carryinformation, e.g., bitrate and resolution, representing the field ofview for encoding, although the present disclosure is not limitedthereto. In this implementation, there may be commands in the metadatato indicate that the video is a forced view and should be displayedregardless of the viewer's actual current viewing angle or cameraposition.

In some other cases, the user may use a mouse to manually select thesmaller view on the screen have a smaller view than the size of thescreen.

The user can change the field of view by turning his/her head to look ata different part of the immersive video. The user can use a mouse,joystick, or other controller to pan/tilt/move the point of view so thatthe field of view changes. The change of the field of view 305 may bedetected by the user device, for example, the user device may detect, byusing eye-tracking technology (as will be described later), a point inthe image that is the focus of the user's eye gaze, and the user devicemay determine a field of view 305 around that point, having the point inthe center of the field of view 305. In some cases, the field of viewmay change when the user moves closer to or farther away from thedisplay screen, although the present disclosure is not limited thereto.Examples of changing field of view will be described below withreference to FIG. 6.

Even though in FIG. 3A the field of view 305 is shown to have arectangular shape as an example, it should be understood that the shapeof the field of view 305 is not limited thereto. Rather, the shape ofthe field of view 305 may be any shape. For example, as discussedpreviously, in a video game where a user's character may look through asight scope of a rifle, the field of view may be round.

Still referring to FIG. 3A, the display may be presenting the user withthe field of view 305, but the user's eyes might be focused on a smallerarea 301 within that field of view. This smaller area 301, an area offocus, may comprise, for example, blocks in the matrix. Since abandwidth for video delivery to a user device may be fixed within acertain period of time, e.g., 4 Megabits/second, to efficientlydistribute the bandwidth for the video frame during the video delivery,the server may allocate a larger percentage (e.g., 70%) of the bandwidthfor transmitting video content associated with the area of focus, andmay allocate a smaller percentage of the bandwidth for transmittingvideo content other than video content associated with the area offocus. The example percentage 70% mentioned above may vary based on oneor more of a degree to which other items of interest appear in thatvideo frame outside the area of focus, or a likelihood of the user toview areas outside of the area of focus. The area of focus 301 isdisposed at the center of field of view 305. As an example, the blocksin the matrix

$\begin{matrix}B_{8,6} & \cdots & B_{10,6} \\ \vdots & \ddots & \vdots \\B_{8,4} & \cdots & B_{10,4}\end{matrix}$

are disposed within or overlap the boundary of area of focus 301, andthus may comprise and be associated with the area of focus 301. The areaof focus 301 may be at the center of the field of view 305 asillustrated in FIG. 3A if, for example, the user happens to be lookingat the center of the display (and if the field of view is based onhead-tracking, and not eye-tracking). Alternatively, in somecircumstances, the area of focus 301 may be offset from the center ofthe field of view 305 if, for example, the user is looking at a cornerof the display. Even though in FIG. 3A the area of focus 301 isillustrated as having an oval shape, it should be understood that theshape of the area of focus 301 is not limited to an oval. Rather, theshape of area of focus 301 may be any shape, and the area of focus 301may be in any location.

In some embodiments, the area of focus 301 may be a region of interestwhere an event occurs. For example, the video editor may embed metadatain a video file to indicate locations in a video frame that should begiven heightened attention (e.g., a movie director wants to make sure anexplosion is delivered in high resolution regardless of whether theuser's attention is focused on the exploding object). The region ofinterest may be automatically designated an area of focus 301 andtreated accordingly.

Alternatively or in addition, the user may select the area of focus 301by drawing a boundary or boundaries for a target area or multiple targetareas on a touch screen.

Alternatively or in addition, eye tracking may be implemented todetermine the area of focus 301 based on movements of the user's eyes.For example, an eye-tracking system can be used to determine a point inthe field of view where the user is looking, and the system can select aregion around that point to be the area of focus. Eye tracking is aprocess of measuring the area of focus by optical tracking.

Single or multiple users may watch the video displayed on the displayscreen of the single user device at the same time. In the case that onlya single user watches the video frame, there may be only one area offocus, and the area of focus 301 may be selected by the user device orthe user, or be determined by eye tracking. The server may receiveinformation representing the selected or determined area of focus anduse the information to encode blocks associated with the area of focusand then transmit the encoded blocks to the user device. In the casethat multiple users watch the video frame on the display screen of thesingle user device at same time, there may be more than one area offocus since different users may focus on different areas of the videoframe. Eye tracking may be used to identify the more than one area offocus.

In some cases, multiple users are watching a same video frame and thevideo frame may have an aggregate area of focus presenting a combinationof areas of focus among the multiple users. Determining the aggregatearea of focus may allow the system to prioritize the areas of focus tomaximize the efficient use of total bandwidth for enhancing each of theareas that the viewers are focusing on. Among the multiple users, eachuser may have an area of focus that is the same as, or different from,an area of focus of another user. The aggregate area of focus may be acommon area of focus among a substantial number of or all of themultiple users. The substantial number of users may refer to a numbergreater than a threshold number, for example, 20% (one-fifth) of theusers. That is, when more than 20% of the users are focusing on acertain area, such as a group of blocks, then that area may bedetermined to be an aggregate area and the server may allocate morebandwidth to transmit blocks of the aggregate area so that the 10% ofthe users may obtain enhanced viewing experience. The server mayallocate less bandwidth to transmit blocks of areas receiving attentionof less than 20% of the users. In some examples, the threshold numberindicated above may be smaller than 20%, for example, may be 10%. Thereare may be multiple aggregate areas of focus for one video frame. In oneimplementation, the common area of focus may be determined to be an areaof focus common to a greatest portion of the user devices. More detailsof determining the aggregate area(s) of focus will be described in FIG.13.

In the above cases where multiple users are watching a same video frameand the video frame has an aggregate area of focus, the server maytransmit (e.g., via multicast transmission) a baseline layer of theentire video frame to a plurality of user devices, and transmit (e.g.,via unicast transmission) a separate stream comprising enhancement layerdata to enhance the respective user's area of focus. In the baselinelayer, the blocks comprising the aggregate area of focus may have higherresolution than other blocks. When the blocks are MBR blocks, during themulticast transmission of the baseline layer, a respective higherbitrate version of each of the blocks comprising the aggregate area offocus may be selected and transmitted among its respective multiplebitrate versions. The enhancement layer may increase resolution of therespective user's area of focus. In the examples discussed previouslywhere the blocks may have different sizes for different regions of thevideo frame, blocks comprising the field of view may be smaller thanthose outside of the field of view to allow finer grained control ofwhich regions of the video frame to get higher bitrate MBR data.

In some examples when multiple users (e.g., two users) may be watching asame video, one of the users may allow the other user to control hisfield of view, and each user may independently have their own area offocus (within the same field of view controlled by the other user). Theserver may multicast (or broadcast) blocks comprising the field of viewcontrolled by the other user to both users, and unicast a respectiveenhancement layer to each user device to enhance the respective area offocus of each user.

In some cases, the system can further help when there are multipleusers. If there are too many users for enhancing each of the areas offocus to be feasible (e.g., if a nationwide broadcast of a live sportingevent is watched by thousands), the system may try to prioritize theareas of focus to maximize the efficient use of the enhancement layer bypredetermining the area of focus based on content of the video frame orbased on audio content associated with the video frame. For example,when an event, e.g., a car explosion, occurs in the video frame, thevideo editor may determine that the user's attention is likely to be, orshould be, attracted by flames of the car explosion; therefore, thevideo editor may predetermine that the blocks presenting the carexplosion scene are the region of interest and are the area of focus. Inthe exemplary event of car explosion, the sound of car explosion mayattract the user's attention and the user may turn his head towards thedirection of the car explosion. The video editor may predetermine thatthe area of focus include video block(s) associated with the audiocontent of the car explosion.

In some cases, the system may determine the areas of focus of a subsetof users who are viewing a video content currently being transmitted(e.g., a live sporting event broadcast, a television episode multicast,etc.), and can use that area of focus to determine what areas andenhancement layers should be sent for other users who are also viewingthe video content. In such an implementation, those other users mayreceive the video content with a delay (e.g., 30 seconds) as compared tothe users in the subset. The duration of the delay may depend on theamount of processing time needed for the system to collect and processthe viewer information of the subset of users, and to prepare thenecessary enhancement layers for the other users.

In some cases where the predetermined area of focus 301 is outside thecurrent field of view, the user has to shift his field of view 305 sothat the predetermined area of focus 301 falls within the new field ofview.

In some cases, the video editor may embed metadata that providesinformation about the area of focus for one or more video frames in avideo. For a computer-generated video frame, the video editor maydetermine the area of focus and may include metadata representing thearea of focus in the video stream. Metadata may include blockcoordinates for corners of the focus area 301 or a listing of the blocksthat comprise the focus area 301. Metadata may also include resolutionpreference for at least some regions of a video frame. For example, thevideo editor may prefer an area of focus to have a higher resolutionthan that of another area outside the area of focus in order to attractviewers' attention to the area of focus. Alternatively, the video editormay prefer an area of focus to have a lower resolution than that ofanother area outside the area of focus, to achieve a blurred effect. Insome cases, field of view 305 can be represented in the metadata in thesame way discussed above for the area of focus 301.

In some cases, the area of interest predetermined by the video editormay be overridden by the aggregate area of focus determined bystatistical data collected from the multiple users. In some cases, theaggregate area of focus determined by statistical data collected fromthe multiple users may be overridden by the area of interestpredetermined by the video editor.

In one example, the video editor may decide that area X should be giventhe focus, but if enough viewers are looking at area Y instead (e.g.,instead of watching a hockey puck, the viewers are looking at a fightthat has broken out on the ice at a different location), then the systemmay provide an enhancement layer for the area of the fight. This may bein addition to, or even instead of, the area predetermined by the videoeditor, and that may occur based on the volume of user viewing. Forexample, if area X is the predetermined area of focus, the system maydecide to transmit an enhancement layer for area Y instead of (or inaddition to) that of area X if some threshold quantity (e.g., 30%) ofviewers are looking at area Y.

The opposite may also be true. In some cases, a video editor may decideto override what the users are actually looking at. For example, if astreaker runs onto a soccer pitch, the video editor may decide that,regardless of what percentage of viewers are looking at the streaker,the system will not provide enhancement layer data for the areacontaining the streaker. This overriding may occur, for example,dynamically in response to a video editor's command entered as theevents of the program unfold.

In some examples, multiple user devices may receive the video frame 110and one or more of the user devices may be digital video recorder (DVR)devices. When a user of a DVR device does not happen to be viewing avideo frame when the DVR is recording, the server may infer a defaultarea of focus, e.g., the aggregate area of focus determined based on allof the areas of focus of the other users, and deliver the video framealong with an enhancement layer applied to the aggregate area. When theuser is watching a video frame and recording the video framesimultaneously, the content provider may deliver the video frame, alongwith an enhancement layer corresponding to the user's area of focus, tothe digital video recorder for display and recording.

Information of field(s) of view and/or area(s) of focus may be used fordelivering a subsequent video frame following the present video frame.

The server described in FIG. 3C may be the push notification server 105and/or the content server 106 and/or application server 107 or othervarious servers or components described above with reference to FIG. 1.The server may reside on a premise that is different from the premise(s)on which the user devices reside.

FIG. 3C illustrates an exemplary schematic diagram of a communicationsystem that may be implemented to deliver a partitioned video frameaccording to one or more illustrative aspects of the disclosure. Forexample, the communication system of FIG. 3C may be used to deliver thepartitioned video frame of FIG. 3A.

In FIG. 3C, a content server 306 may have similar structure and mayperform similar functions as the content server 106 illustrated inFIG. 1. An application server 307 may have similar structure and mayperform similar functions as the application server 107 illustrated inFIG. 1. A user device 366 may be a device configured to receive and/ordisplay video content. For example, the device 366 may be a virtualreality headset, and a user may be watching a 360-degree movie whilewearing the virtual reality headset, and may be viewing the movie on theheadset's display. Although shown separately in FIG. 3C, all or aportion of the functions of the content server 306 and the applicationserver 307 may be combined. Even though only one user device isillustrated in FIG. 3C, it should be understood that multiple userdevices may be connected to the application server 307 by a link 310.

As shown in FIG. 3C, the content server 306 may transmit a source videoframe to an encoder 3072 of the application server 307. The encoder 3072may partition the received video frame into multiple blocks. In somecases, the encoder 3072 may partition the video frame into blocks (e.g.,MBR blocks) having different shapes and difference sizes, and may groupsome discrete blocks into as an entirety for encoding. The encoder 3072may encode the blocks each having multiple bitrate versions and storesthe multiple bitrate versions of each block for transmission based oninformation representing a field of view (or fields of view) and an areaof focus (or areas of focus) that may, for example, be determined by theuser of the user device, by statistical data received by multiple users,or by the video editor without relying on the statistical data. Theencoder 3072 may receive information representing field(s) of view andarea(s) of focus from a processor 3666. For example, for the video frame30 of FIG. 3A, the encoder 3072 may encode blocks associated with thefield of view 305 for transmission to the user device using a firstbitrate. In this example, the encoder might not encode all of the 20×8blocks of the video frame 30, because the blocks associated with thefield of view 305 may be sufficient to fill the user's view, andbandwidth may be saved by transmitting a portion of the blocks of thevideo frame 30. In some cases, the user's vision may degrade from thearea of focus toward the areas outside the area of focus, and resolutionrequirement may drop from the area of focus toward the areas outside thearea of focus. Thus, the encoder 3072 may encode the blocks associatedwith the area of focus 301 for transmission to the user device using asecond bitrate that is higher than the first bitrate, and encode theblocks associated with the field of view 305 (expect for those blocksassociated with the area of focus 301) for transmission to the userdevice using the first bitrate. As an alternative to encoding the blocksassociated with the area of focus 301 using the higher second bitrate,the encoder 3072 may encode the field of view 305 using the firstbitrate and apply an enhancement layer to increase resolution of thearea of focus 301. The enhancement layer may be scalable video codec(SVC) enhancement layer for MPEG-4 standard. It should be understoodthat the enhancement layer is not limited to SVC enhancement layer andmay be any suitable enhancement layer that can increase resolution of atarget area of the video frame. Each block may be encoded independentlyfrom each other or may be encoded dependently with one or more blocks.The encoded blocks may be transmitted to one or more user devices by theserver.

The encoder 3072 may transmit the encoded blocks to a transceiver 3073of the application server 307. The transceiver 3073 may communicate witha transceiver 3663 of the user device 366 via the link 310 and maytransmit the encoded blocks to the user device 366. Decoder 3662 of theuser device 366 may receive and decode the encoded blocks, and maydisplay the video frame with all or a portion of the decoded blocks on adisplay 3665.

In some cases, the encoder 3072 may encode each block in multipleresolutions (which may use different bitrates) and transmit theseversions to the transceiver 3073. Then, based on the field of view, thearea of focus of the user of the user device 366, and/or an aggregatearea of focus as previously discussed, the transceiver 3073 may selectblocks to be transmitted to the user device 366.

In some cases where multiple user devices consume a video frame andusers of the multiple user devices have overlapping fields of view, theserver may multicast the overlapped portion (possibly with higherresolution) to the multiple user devices in order to improvetransmission efficiency and optimize network bandwidth usage. FIG. 4illustrates an example of fields of view in a video frame according toone or more illustrative aspects of the disclosure.

As shown in FIG. 4, more than one user device may view the video frame,and the fields of view of these user devices may have one or moreoverlapped area(s). The fields of view 405 and 408 of the first andsecond user devices U1 and U2 have an overlapped portion A (e.g., theshaded portion in FIG. 4). In one implementation, while startingseparate unicast transmissions for the fields of view 405 and 408 of thefirst and second user devices U1 and U2, the server may provide amulticast stream that both devices U1 and U2 join, and may put theblocks comprising the overlapped area A and optionally the areas thatare outside a combination of areas A, B, and C in the multicast stream.In this implementation, the server may unicast transmissions for therespective non-overlapping portions B and C of the respective fields ofview to the respective user devices U1 and U2. Blocks associated withthe overlapped portion A may refer to blocks disposed completely withinthe overlapped portion A, partially within the overlapped portion A, orcombinations thereof. In one example, the blocks associated with theoverlapped portion A may additionally include blocks of which theboundaries are overlapped with the boundary of the overlapped portion Aeven if the entirety of such blocks are outside the overlapped portionA. In this example, the server may determine the overlapped portion A asa common field of view which represents an area viewable to multipleusers, and may apply an enhancement layer, e.g., an SVC enhancementlayer, onto the overlapped portion A in order to increase resolution ofthe overlapped portion A.

In the example of FIG. 4, a field of view 405 of a video frame 40 for afirst user device U1 is configured to be larger in size compared to afield of view 408 of the video frame 40 for a second user device U2.This may be due to user device U1 being physically larger than userdevice U2 such that it can fit more of the image on screen, or it may bedue to the first user device U1 being zoomed in to the image a bit lessthan the second user device.

Even though fields of view 405 and 408, as shown in FIG. 4, partiallyoverlap with each other, it should be understood that this is exemplaryand that the fields of view 405 and 408 may be located differently withrespect to each other. For example, the fields of view 405 and 408 maybe spaced apart from each other, or the field of view 408 having arelatively smaller size may be disposed completely within the field ofview 405.

In some implementations where different users may use display deviceswith different display capability, the server may deliver the videoframe based on the different display capabilities. For example, thefirst user device U1 is a 16 k display and the second user device U2 isa 4 k display. The fields of view 405 and 408 of the users of the userdevices U1 and U2 overlap as shown on FIG. 4. The server may eithermulticast (or broadcast) a baseline, which includes all blocks of thevideo frame 40, to the user devices U1 and U2; or multicast (orbroadcast) a baseline, which includes blocks comprising regions A, B,and C but excludes blocks outside the combinations of regions A, B, andC. In case that the server multicasts a baseline, which includes allblocks of the video frame 40, to the user devices U1 and U2, the blocksoutside the combination of the regions A, B, and C may have a lowerresolution, for example, 1080 p. The server may multicast to the userdevices U1 and U2 an enhancement layer that includes blocks comprisingthe overlapped portion A and increase the resolution of the overlappedportion A from 1080 p to 4 k. Thus, the multicast stream from the servermay include the baseline including all blocks of the video frame 40 andthe enhancement layer that includes blocks comprising the overlappedportion A. The server may unicast to the second user device U2 anenhancement layer that includes blocks comprising the portion B toincrease the resolution of the portion B from 1080 p to 4 k. Thus, theunicast stream from the server to the second user device U2 includes theenhancement layer including blocks comprising the portion B. The servermay unicast to the first user device U1 an enhancement layer thatincludes blocks comprising the region A and further increase theresolution of region A from 4 k to 16 k. The server may unicast to thefirst user device U1 an enhancement layer that includes blockscomprising the region C and increase the resolution of region C from1080 p to 16 k. Thus, the unicast stream from the server to the firstuser device U1 includes the enhancement layer including blockscomprising the region A and the enhancement layer including blockscomprising the region C.

In some implementations, the server may package multiple blocks as agroup to represent a certain object, for example, a building, and mayencode and transmit the group as an entirety. For example, the servermay encode one group of blocks in a same video file. This may simplifyencoding and expedite transmission since multiple blocks may be encodedand transmitted together rather than individually. FIG. 5 illustrates anexample in which blocks of a partitioned video frame are packaged as agroup according to one or more illustrative aspects of the disclosure.

In the example of FIG. 5, the encoder 3072 may encode ungrouped blocksindependently from each other. In a video frame 50, three differentgroups G1, G2, and G3 are packaged with their respective blocks. Eventhough only three groups are shown in FIG. 5, the number of groups mayvary. For example, the number of groups may be determined by the videoeditor. Group G1 includes 8 blocks and may represent an object, e.g., abuilding. Group G2 includes 4 blocks and may represent another object,e.g., a tree. Group G3 includes 4 blocks, among which some blocks arediscretely distributed, and may represent a discontinuous object. Itshould be understood that any number of blocks may be included in agroup, and that all the blocks of the same group may represent acontinuous region in the video frame or some blocks of one group may bein one region separated from another region represented by other blocksof the same one group.

The encoder 3072 may encode groups G1, G2, and G3 for transmission usingbitrates R_(G1), R_(G2), and R_(G3), respectively. As such, during thetransmission of groups G1, G2, and G3, the data stream has differentrates of conveying bits at different times. As an example, larger groupsmay be encoded for transmission using higher bitrates. As anotherexample, a group disposed entirely or partially within the area of focusmay be encoded for transmission using a higher bitrate. Still as anotherexample, groups completely outside the area of focus may be encodedusing lower bitrates. The shapes of the groups are not limited to thosedepicted in FIG. 5 and the groups may have any other shapes.

In some cases, the server may create a number of different resolutionversions for each group of blocks and each ungrouped block, and eachresolution version of each group and ungrouped block may be separatelyencoded in a certain type of compressed format, e.g., MPEG-4 format. Assuch, the server may reassemble groups of blocks and ungroup blocks, fortransmission, with any of these resolution versions at a later point oftime, based on the field of view and the area of focus.

FIG. 6 illustrates an example in which a field of view in a video frameis changed for a user in a time period according to one or moreillustrative aspects of the disclosure. In this example, when a userwearing a VR headset views a 360 degree video, the user turns his heador moves his eyes from one region to the upper right corner thereofduring a time period from time T1 to T2 to T3. During the movement ofthe eyes, the field of view may change from 601 (T1) to 602 (T2) to 603(T3), and a sequence of video frames and blocks associated with thefield of view may change accordingly. The VR headset may updateinformation representing the field of view and may send the updatedinformation to the server. The server may use the updated information todetermine one or more blocks associated with the current field of view,encode the one or more blocks for transmission, and transmit the one ormore blocks to the VR headset. In some examples, when a user sits closeenough to a television and watches a video, and the user moves eyes orturns his head, the field of view may change accordingly. It should beunderstood that the user device is not limited to a VR headset or atelevision, and may be any other display device or devices. In someexamples, when the user turns his head, the server may identify a newset of blocks that are needed for display. In the case of a static imagebeing viewed, the server may determine whether any needed blocks havealready been received, determine additional blocks that should beincluded in a multicast stream with a lower-resolution for displayingimages within the new field of view, and then issue unicast requests forenhancement layer versions of blocks comprising a new area of focuswithin the new field of view.

FIG. 7 illustrates an example of delivering groups of blocks when afield of view in a video frame is changed according to one or moreillustrative aspects of the disclosure. When the field of view changes,blocks packaged as a group may be delivered to the user device even ifsome of blocks of the group might not be included in the field of view.For convenience of explanation, it is assumed that in FIG. 7, therelative position of each group in one video frame is the same as insubsequent video frames (in other words, the video remains the same fromframe-to-frame in this example, and the only thing changing is theuser's focus).

In reference to FIG. 7, the field of view of a user changes from 701 to702 to 703 during the time period from time T1 to T2 to T3. The VRheadset may update information representing the field of view and maysend the updated information to the server. At time T2, even thoughblocks B_(13,4) and B_(13,5) of group G2 are not associated with thefield of view 702, the server might continue to deliver all blocks ofgroup G2 to the user device in consideration that blocks B_(12,4) andB_(12,5) of group G2 are still associated with the field of view 702.Similarly, at time point T3, the server may receive updated informationand determines that blocks

$\begin{matrix}B_{8,6} & B_{9,6} & B_{10,6} \\B_{8,5} & B_{9,5} & B_{10,5}\end{matrix}$

of group G1 are associated with the field of view 703. In response to adetermination that at least one block of group G1 is associated with thefield of view, the server may continue to deliver all blocks of group G1to the user device. In the case that none of blocks of a group areassociated with the field of view, the server may stop delivering thegroup as an entirety to the user device. In this circumstance, the userdevice may no longer get an enhancement layer previously applied forthis group, and the user device would resort to displaying thelow-resolution multicast version of the blocks in this group. The servermay encode each group independently from each other. That is, group G1may be encoded for transmission using bitrate R_(G1), group G2 may beencoded for transmission using bitrate R_(G2), and group G3 may beencoded for transmission using bitrate R_(G3). Bitrate R_(G1), bitrateR_(G2), and bitrate R_(G3) may be the same as or different from eachother. One of ordinary skill in the art would recognize that theungrouped blocks may be encoded independently from each other andindependently from any grouped blocks.

FIG. 8 illustrates an example of encoding each block of a partitionedvideo frame independently from other blocks according to one or moreillustrative aspects of the disclosure. The server may selectivelytransmit any one of the encoded blocks at a respective predeterminedbitrate. Referring to FIG. 8, each block B_(i,j) of a video frame 80 isencoded independently from the other blocks. Each block B_(i,j) may beencoded for transmission using a respective bitrate R_(i,j). A bitratemay be selected for one block, with or without consideration of thebitrate of its neighboring blocks. In some cases, each block may beencoded as an individual video file. In one implementation, the servermay create a number of different versions for each block and eachversion of each block may be separately encoded in a certain type ofcompressed format, e.g., MPEG-4 format. In this implementation, oneversion of a block may have a bitrate different from another version ofthe block.

In some cases, the server may create a number of different versions foreach group of blocks and each ungrouped block, and each version of eachgroup and ungrouped block may be separately encoded in a certain type ofcompressed format, e.g., MPEG-4 format. The video editor may reassemblegroups of blocks and ungroup blocks with any of the available versionsat a later point of time.

FIG. 9 illustrates an example of encoding a video frame in which anevent occurs according to one or more illustrative aspects of thedisclosure. For convenience of explanation, in FIG. 9, video frames arepartitioned in the same manner as each other, although the presentdisclosure is not limited thereto. In this example, at time T1, a userusing a VR headset may view images within a field of view V1 with anarea of focus at a central portion of the field of view V1. At time T2,an event (e.g., a car explosion) may occur at block B_(15,5). The carexplosion may attract the attention of the user, and the user mayrespond by moving their areas of focus onto block B_(15,5) and itsperipheral (or neighboring) blocks. The field of view of the user mayshift from V1 to V2 accordingly. The user device may update theinformation presenting the field of view and the area of focus (e.g., anupdated list of blocks of the field of view and an updated list ofblocks of the area of focus, or simply an updated list of coordinatesindicating where the user is looking, if the field of view and area offocus are processed based on where the user is looking), and may sendthe updated information to the server. The server may determine whetherthe user will be still looking at block B_(15,5) in the next videoframe. If it is determined that the user will be still looking at blockB_(15,5), the server may select a higher bitrate version, than that ofthe previous video frame, among the stored multiple bitrate versions fortransmission in order to provide a better resolution for B_(15,5). Insome examples, compared to the bitrates of other blocks of the previousvideo frame that have same indices, bitrates of the respective otherblocks may be either maintained or decreased so that resolution of theother blocks is lower than the resolution of B_(15,5). The server maydeliver blocks associated with the field of view V2 to the user device.Even though in this example, the car explosion event is confined withinone block, it should be understood that an event may expand into morethan one block.

In order to account for anticipated movement by the user, the system cantransmit slightly more than just the blocks in the field of view 305.That way, if the user moves his/her head a little, the neighboringblocks will already have been provided and may be quickly displayed. Tosolve such problem, while transmitting two successive video frames,especially those having substantially same content with only smalldifferent details, the server may transmit additional column(s) and/orrow(s) surrounding the field of view to the user device. An example ofthese cases will be described in FIG. 10. FIG. 10 illustrates an exampleof delivery of a partitioned video frame according to one or moreillustrative aspects of the disclosure. Extent of the coverage ofadditional blocks as discussed above may depend on motion estimationand/or other empirically available characteristics. For example, forvideos of some sport games which have larger variation horizontally butless vertically, the number of additional columns of blocks may begreater than the number of additional rows of blocks.

In the example of FIG. 10, when a user wearing, for example, a VRheadset turns their head, the field of view changes from 1010 to 1020from time T1 to time T2. At time T1, in addition to delivery of blocks

$\begin{matrix}B_{6,7} & \cdots & B_{12,7} \\ \vdots & \ddots & \vdots \\B_{6,3} & \cdots & B_{12,3}\end{matrix}$

associated with the field of view 1010, the server may deliver blocksB_(5,2), B_(5,3), . . . B_(5,8), B_(6,2), B_(7,2), . . . B_(13,2),B_(5,8), B_(6,8), . . . B_(13,8), and B_(13,3), B_(13,4), B_(13,7) whichare disposed outside of and neighboring the blocks associated with fieldof view 1010 (within the area 1015). In some of examples, blocksB_(5,2), B_(5,3), . . . B_(5,8), B_(6,2), B_(7,2), . . . B_(13,2),B_(5,8), B_(6,8), . . . B_(13,8), and B_(13,3), B_(13,4), . . . B_(13,7)may be encoded for transmission using a bitrate lower than the blocks

$\begin{matrix}B_{6,7} & \cdots & B_{12,7} \\ \vdots & \ddots & \vdots \\B_{6,3} & \cdots & B_{12,3}\end{matrix}$

associated with field of view 1010. At time T2, when the field of viewchanges to field of view 1020, blocks B_(7,2), B_(8,2), . . . B_(12,2)and B_(13,2), B_(13,3), . . . , B_(13,6) whose indices were notassociated with the previous field of view 1010 may be requested fordisplay to the user. When two successive video frames have substantiallysame content with only small different details, since blocks B_(7,2),B_(8,2), . . . B_(12,2) and B_(13,2), B_(13,3), . . . , B_(13,6) of theprevious frame 1010 may have already been delivered to the user deviceat time T1 and have been available for being reviewed since time T1,consistent visual quality may be achieved and the user might still seevideo content (rather than blank images) at blocks B_(7,2), B_(8,2), . .. B_(12,2) and B_(13,2), B_(13,3), . . . , B_(13,6)) when the user turnshis head and changes to the field of view 1020 at time T2. In someexamples, blocks associated with the field of view 1020 and blocksneighboring to the blocks associated with the field of view 1020 may beencoded for transmission using a low bitrate, and an enhancement layermay be applied to the field of view. At time T2, the server may deliverblocks within the area 1025 to the user device. Although in the exampleillustrated in FIG. 10 one additional column and/or one additional rowof blocks immediately neighboring blocks associated with the field ofview 1010 or 1020 are transmitted to the user device, it should beunderstood that any number of columns and/or rows of blocks may betransmitted to the user device. Even though two successive video framesare described above as an example, the above described could insteadhappen between two successive video fragments.

In some cases, when additional blocks outside of the field of view aretransmitted, the server may allocate bandwidth between the blockscomprising the area of focus, the blocks comprising the field of view,and the additional blocks. An example of these cases will be describedin FIG. 11. FIG. 11 illustrates an example of delivery of a partitionedvideo frame with an enhancement layer according to one or moreillustrative aspects of the disclosure.

In the example of FIG. 11, blocks associated with field of view 1105,and blocks neighboring the blocks associated with field of view 1105,may be delivered to the user device. That is, blocks disposed withinboundary of 1110 may be delivered to the user device. The blocksassociated with the field of view 1105 and the blocks neighboring theblocks associated with field of view 1105 may be encoded fortransmission. An enhancement layer may be encoded and transmitted to theuser device to improve resolution of the blocks associated with field ofview 1105. In some examples, another enhancement layer may be encodedand transmitted to the user device to further improve resolution of thearea of focus 1101. In this example, the user may be provided with highresolution video content at the area of focus 1101.

In some examples, multiple user devices may request a video frame fordisplay, and the server may determine how to deliver the video frame tothe multiple user devices based on data transmitted from the userdevices, e.g., fields of view and areas of focus, and/or metadataincluded in the video file. In order to efficiently deliver the videoframe to the multiple user devices, the server may multicast blocksassociated with the aggregate area of focus and/or the common field ofview to user devices and unicast blocks associated with the respectiveun-overlapped areas of the respective field of view and/or field of viewto the respective user device. An example of delivering a video frame tomultiple user devices will be described in FIG. 12. FIG. 12 is asignaling diagram of a method for video content delivery to multipleuser devices according to one or more illustrative aspects of thedisclosure.

In the example of FIG. 12, when a user (the user of either the userdevice 1210 or 1220) starts to watch video, the server may transmit theinitial video frame(s) based on at least one of the metadata thatindicates location(s) in the video frame(s) that should be givenheightened attention, the pre-stored aggregate area of focus and/orcommon field of view which have been stored based on statistical datacollected from other users who have watched at least a portion of thevideo content, or other pre-stored criteria that allow the server todetermine how to transmit the initial video frame(s). During theconsumption of the initial video frame(s), the user device has trackedthe field of view and the area of focus of the user. These steps are notillustrated but may be performed before step S1222.

At step S1222, the user of the user device 1220 has consumed the initialframes and continues to consume the rest of the video content, and theuser device 1220 may transmit parameters, e.g., information indicatingtype of display, information indicating the previously tracked field ofview, and information indicating the previously tracked area of focus,to the server, e.g., at least one of the content server 306 and theapplication server 307. As discussed above, the information of field ofview may include regions on the display that the user can see at a givenmoment in time, and the information of the area of focus may includeregions on the display that the user is focused on.

At step S1232, the server may receive these parameters transmitted atstep S1222, and register these parameters for user device 1220 in adatabase.

At step S1212, similarly to the user device 1220, the user device 1210may transmit parameters, e.g., information of type of display,information of field of view, and information of area of focus, to theserver.

At step S1234, the server may receive the parameters transmitted at stepS1212, and register these parameters for user device 1210 in thedatabase.

It should be understood that, before the video is transmitted to theuser devices, the server may partition each of the video frames includedin the video into blocks, create multiple versions of resolutions foreach block of each of the video frames, and store these versions of theblocks in a database for future use in delivering the video.Alternatively, the server may perform the partitioning and encoding in areal-time fashion, e.g., partitioning and encoding a video frame uponthe video frame being requested for access. At step S1236 in the exampleof FIG. 12, the server partitions a video frame and create multipleversions for each block before upon the video frame is requested foraccess by both of the user devices 1210 and 1220.

In step S1236, the server may determine the aggregate area of focusbased on information of area of focus received by the user devices 1210and 1220. As discussed previously, the server may create a baselinelayer of the entire video frame for transmission (e.g., via multicasttransmission which will be described in steps S1251 and S1261) to theuser devices 1210 and 1220, and create a separate stream comprisingenhancement layer data for transmission (e.g., via unicast transmissionwhich will be described in steps S1253) to enhance the respective user'sarea of focus. During the creation of the baseline layer, the sever mayselect, for transmission, one of bitrate version for each block, and mayselect a higher bitrate version for blocks disposed within the aggregatearea of focus than blocks disposed outside the aggregate area of focus.

In some cases, the server may determine the aggregate area of focusbased on the metadata. For example, the metadata may have indices ofblocks including content of an event. In these cases, the server mayinclude a higher bitrate version for the blocks including the content ofthe event in the baseline layer and include a lower bitrate version forblocks not including the content of the event in the baseline layer.

In some cases where the fields of view of the users of the user devices1210 and 1220 do not comprise all blocks of the video frame, thebaseline layer may only include blocks within either of the fields ofview and not include blocks completely outside either of the fields ofview.

In some cases, the server may determine any change of aggregate or localarea of focus for the users of the user devices 1210 and 1220 and mayupdate indices of blocks of the next frame to be transmitted based onsuch change.

In some cases, the server may perform the partitioning and encoding canbe done prior to a request to view the content.

It should be understood that the server may encode the video frame byany other methods that have been described or will be described in thedisclosure and that have been known as necessary procedures during anencoding process.

At step S1237, the server may store data representing common field ofview and aggregate area of focus for future reference. The data mayinclude indices of the blocks associated with the common field of viewand indices of the blocks associated with the aggregate area of focus.

At step S1251 and step S1261, the server may multicast some or all ofthe blocks to the user devices 1210 and 1220. This multicasttransmission may include the entire frame 30, or in some embodiments,the multicast transmission may only include the blocks in theoverlapping field of view between the user devices 1210 and 1220 (withperhaps the additional neighboring blocks discussed above). In someembodiments, this multicast transmission may contain the overlap blocksin a baseline resolution, and may include enhancement layer informationas well.

In step 1253, the server may transmit a unicast transmission to the userdevice 1210. The unicast transmission may contain enhancement layerinformation for the block(s) that are in the first user's area of focus,minus any enhancement layer information that may have been included inthe multicast transmission above. A similar unicast transmission may bemade for the user device 1220, containing enhancement layer informationfor the blocks in the second user's area of focus. However, in FIG. 12,the user device 1220 is not shown as receiving this unicasttransmission, and this may occur if, for example, the user of thatdevice happened to have a small focus area whose enhancement layerinformation was already included in the multicast transmission receivedabove.

At step S1214, the user device 1210 may receive the multicast blocks andunicast blocks from the server, decode the received blocks, and displayvideo content of the decoded blocks. At step S1224, similarly to theuser device 1210, the user device 1220 may receive the multicast blocksfrom the server, decode the received blocks, and display video contentof the decoded blocks.

When the user of the user device 1210 changes his area of focus, theuser device 1210 may send information, at step S1216, representing hisupdated area of focus to the server. At step S1238, upon receiving theinformation from the user device 1210, the server may register theupdated area of focus for the user device 1210 and update the aggregatearea of focus based on the changed area of focus of the user.

At step S1240, based on data representing the updated aggregate area offocus, the server may partition a subsequent video frame into blocks andencode the blocks. Description above with respect to step S1236 may beapplied to step S1240.

At steps S1255 and S1263, the server may multicast blocks associatedwith the updated aggregate area of focus to both of the user devices1210 and 1220. In some examples, these multicast streams for the blocksassociated with the updated aggregate area of focus may replace theprevious multicast streams for the blocks associated with the previousaggregate area of focus.

At step S1265, the server may unicast to user device 1220 an enhancementlayer applied to the area of focus of the user of user device 1220 basedon a determination that the area of focus of the user of user device1220 is different from the updated aggregate area of focus.

At step S1218, the user device 1210 may receive the multicast blocksfrom the server, decode the received blocks, and display video contentof the decoded blocks. At step S1226, the user device 1220 may receivethe multicast blocks and/or unicast blocks from the server, decode thereceived blocks, and display video content of the decoded blocks. Duringthe decoding the received blocks, the user device 1210/1220 mayrecombine the received blocks into the original image by using thecoordinates in the horizontal and vertical indices of each block.

It should be understood that the above steps can be repeated as long asthe users of the user devices 1210 and 1220 continue to watch theprogram.

Even though two user devices are illustrated in FIG. 12, it should beunderstood that any number of user devices may be included in thecommunication system. In some examples, the server may registerinformation indicating a type of display for more than two user devicesand may divide the user devices into groups having a same type ofdisplay. When the server determines that multiple user devices are usinga same type of display, the server may group the multiple user devicesand multicast blocks associated with the field of view of this type ofdisplay to the group. The server may additionally multicast blocksneighboring the blocks associated with the field of view to the group.When the server determines that only a single user uses a particulartype of display, the server may unicast blocks associated with the fieldof view of this particular type of display to the single user. Theserver may additionally unicast blocks neighboring the blocks associatedwith the field of view to the single user. In some examples where thetype of display of a group is a wired headset, the users of this groupview a 360 degree video with the wired headsets, and viewable angles ofthe users are less than 360 degrees, e.g., 240 degrees. The server maydetermine to deliver only 240 degree, rather than 360 degree, of thevideo content to this group of user devices.

In some examples, at step S1222 or S1212, the user device mayadditionally transmit a vision profile of the user to the server. Whenthe user's vision profile shows that the user's vision exceeds apredetermined threshold (e.g., a visual acuity of 20/20), the server mayunicast an additional enhancement layer applied to the area of focus ofthe user to the user device. By doing so, the resolution of the user'sarea of focus may be increased to account for the user's strongeyesight. In some examples, when the user's vision profile shows thatthe user's vision is below a predetermined threshold (e.g., a visualacuity of 20/100), the server may determine not to transmit enhancementlayer(s) corresponding to blocks within the area of focus becausedetails requiring higher resolution presentation might not be seen bythe user anyway because of his poor vision.

It should be understood that the steps in the signaling diagramillustrated in FIG. 12 need not all be performed in the order specifiedand some steps may be omitted, changed in order, or performedsimultaneously. In one example, step S1212 may be performed beforeS1222. In a second example, step S1251 and step 1253 may be performedsimultaneously; and step S1263 and step 1265 may be performedsimultaneously. In a third example, step S1214 may occur before or afteror simultaneously with step S1224; and step S1218 may occur before orafter or simultaneously with step S1226. In a fourth example steps S1216and S1238 may be omitted if the user of the user device 1210 does notchange his area of focus. In one or more embodiments, the methodillustrated in FIG. 12 or one or more steps thereof may be performed byone or more computing devices or entities. The method illustrated inFIG. 12, or one or more steps thereof, may be embodied incomputer-executable instructions that are stored in a computer-readablemedium, such as a non-transitory computer readable medium.

FIG. 13 illustrates an example of determining an aggregate area of focusbased on statistical data according to one or more illustrative aspectsof the disclosure. The statistical data includes informationrepresenting common areas of focus of all of or at least some users whoare viewing or have viewed the video frame. In this illustration, aftermultiple users have watched a video, the server determines that eachblock of each video frame 130 of the video receives a percentage ofusers' attention from the multiple users viewing the video frame 130.For example, block B_(15,4) receives 10.2% of users' attention, and thismeans that, out of every 1000 users, 102 users are focusing on blockB_(15,4). Since the percentage of attention from all users that blockB_(15,4) receives is significantly higher than its neighboring blocks,e.g., B_(14,3), B_(15,3), B_(16,3), B_(16,4), B_(16,5), B_(15,5),B_(10,5), and B_(14,4), the server may determine that the area thatB_(15,4) covers is an aggregate area of focus #1. The server mayidentify an aggregate area of focus by comparing a percentage ofattention of a block to percentages of attention of neighboring blocks.For example, an aggregate area of focus may be determined when a blockreceives a percentage of attention that is greater, by a predeterminedamount (e.g., 6%), than a percentage of attention received by any of theblock's neighboring blocks. Alternatively or in addition, the server mayidentify an area of focus when a group of blocks receives a percentageof attention significantly greater, by a predetermined amount (e.g.,4%), than its neighboring blocks by a predetermined percentage. Forexample, if an average percentage of attention of each block of a matrixof blocks

$\begin{matrix}B_{6,5} & \cdots & B_{8,5} \\ \vdots & \ddots & \vdots \\B_{6,3} & \cdots & B_{8,3}\end{matrix}$

is greater than a percentage of attention received by any of itsneighboring blocks by a predetermined percentage, the server mayidentify the area covered by such group as an aggregate area of focus#2. Once the aggregate areas of interest are identified, the server mayregister information representing the aggregate areas of interest forthis video frame for future reference. Even though in the example ofFIG. 13, two aggregate areas of focus are identified, it should beunderstood that any number of aggregate areas of focus may identified.For example, a first aggregate area of focus with the greatest number ofthe area of focus common to all the user devices may be determined, thesecondary aggregate area of focus (the second greatest number of thearea of focuses common to all the user devices) may be determined, thethird aggregate area of focus (the third greatest number of the area offocuses common to all the user devices) may be determined, and so on.

In some cases, in order to optimize usage of the storage of the server,the server may edit bitrate versions of the video frame stored in thestorage of the server. In these cases, before a video frame has beentransmitted for the first time or has been initially transmitted to userdevices a number of times (e.g., 100 times, if the viewing analysisshows statistically significant trends after 100 times of views), theserver may store information representing one or more bitrate versions(e.g., a lowest possible bitrate to a highest possible bitrate andcertain bitrate points in between) for each block based on metadataincluded in the video file and provided by the video editor. Duringdelivery of the video frame for the initial number of times, the servermay deliver blocks with one of the predetermined or received bitrateversions. After delivering the video frame for the initial number oftimes, the server may receive statistical data from user devices and maymodify the bitrate versions by deleting certain bitrate versions basedon the received statistical data. Statistical data may include users'selections of bitrate versions, users' satisfaction of resolution ofdifferent bitrate versions, or other statistical data. For example,after the initial number of times delivery, when the statistical dataindicates that a bitrate version of a block has received a percentage ofusers' attention that is lower than a minimum required percentage (e.g.,0.1%, if the viewing analysis shows statistically significant trendsbased on the 0.1% of users' attention), the server may delete thisbitrate version from the storage of the server. In some cases, thedeleted bitrate version may be moved to and stored in a storage of ahigher level server. If one block has more than one optimized bitrateversion, the server may rank the more than one optimized bitrate versionbased on users' past view requests and may select the most optimalbitrate version for the block for future delivery. Similarly, each groupof blocks may be assigned with a most optimal bitrate version. With theblock(s) of the video frame being further delivered to the user devices,more statistical data may be received by the server from the userdevices and thus the bitrate versions may be further updated for thevideo frame in a similar manner as in the delivery of the video framefor the initial number of times.

In some cases, in order to further optimize usage of the storage of theserver, the server may edit the permanent copy of the video stored inthe storage of the server so that only the blocks that are seen by theusers are kept in the storage of the server, and other blocks that arenever seen by the users are deleted from the storage of the server. Asin the example of FIG. 13, statistical data (e.g., 0%) shows that blockB_(20,1) has not been viewed by any users, over a predetermined timeperiod (e.g., 3 years). Based on the statistical data, the server mightnot encode blocks that have not been viewed or are infrequently viewed,such as block B_(20,1), because it is unlikely that these blocks will bedelivered, or delivered frequently, to any users. Even though blockB_(20,1) is deleted from the storage of the server, this block may bekept in a storage of a higher level server, so that in the case thatthis block is eventually requested by a user(s), the higher level servermay provide this block to a user(s).

In some cases, in order to improve the user's satisfaction, the servermay determine whether the video editor should recreate certain scenesbased on the statistical data. For example, after one or more blocks(and/or one or more groups of blocks) have been multicast with the mostoptimal bitrate for a certain time period, users may provide feedbackindicating that the most optimal bitrate version for one or more blocks(and/or one or more groups of blocks) does not satisfy at least some ofthe users. For example, some of the users may complain that an area ofabout 5 cm² at the central portions of several sequentially displayedvideo frames (from time 1:20:00 to 1:20:15) is not clear. The server mayreceive such feedback and forward the feedback to the video editor. Thevideo editor may either reshoot the scene covering the one or moreblocks (and/or one or more groups of blocks) in a desirable resolution,or modify or recreate the one or more blocks (and/or one or more groupsof blocks) with a computer generated model for the one or more blocks(and/or one or more groups of blocks) to provide a better viewingexperience (e.g., better resolution) to the users. Referring back toFIG. 8, even though each block is shown to be encoded, it should beunderstood that, in some implementations, not every block will beencoded.

In some examples, video content may be associated with non-video content(e.g., audio content comprising stereo audio, 5.1 audio, object basedaudio, ambisonic audio, alternative audio, etc.). Object based audio andambisonic audio may provide 3D sound effect to users. These techniquesassociate the audio content with virtual locations of the objects shownin a video, and the speaker setup of the user uses the associationbetween the audio content and the virtual locations of the objects todetermine how to present the sound encoded in the audio content. Forexample, for a video showing a car explosion in a plaza, the audiocontent encoded with the exploding sound of the car is associated withthe virtual location of the exploded car. The speaker setup of the userreceives the audio content and determines that the exploding sound ofthe car should be transmitted from the virtual location of the explodedcar. The speaker setup then determines how each speaker contributes toproduce the exploding sound, and each speaker produces respective soundbased on such determination. When a video streaming segment is selectedto be transmitted to the user(s), the corresponding a streaming segmentmay be downloaded with any one of the various bitrates independent ofbitrate with which the corresponding video streaming segment istransmitted. When the non-video content is encoded with the videocontent in a common data stream, the non-video content may be an encodedindexed non-video file which is divided into multiple non-video MBRblocks for each video frame. For one video frame, each non-video MBRblock corresponds to a respective video MBR block, and this means thatvideo content represented by each video block has correspondingnon-video content represented by a non-video MBR block. These featureswill be discussed with reference to FIG. 14.

FIG. 14 illustrates an example of non-video content associated with avideo frame according to one or more illustrative aspects of thedisclosure. In this example, in video frame 140, each video blockB_(i,j) corresponds to a non-video block B′_(i,j) including non-videoinformation for the respective block B_(i,j). In order to show that eachvideo block B_(i,j) has its corresponding non-video content, thecorresponding non-video content is illustrated as being included in anon-video block B′_(i,j) and the non-video block B′_(i,j) is illustratedto completely overlap with the corresponding video block B_(i,j).

In some cases, some non-video blocks may be encoded for transmissionusing a same bitrate. In some other cases, each non-video block may beencoded for transmission using a bitrate different from that of any oneof the other non-video blocks. For example, non-video block B′_(10,3) isencoded for transmission using a bitrate R′_(10,3) that may be differentfrom those of other non-video blocks, or alternatively, may be same asthat of one or more of the other non-video blocks. When multiple videoblocks are packaged into a group, the corresponding non-video blocks maybe packaged in a corresponding group. Similarly to a group of videoblocks, the group of non-video blocks may be encoded and transmitted asan entirety. Even though each video block B_(i,j) corresponds to arespective non-video block B′_(i,j), it should be understood that insome examples where some portions of a video frame are not associatedwith any non-video content, one or more video block could have nocorresponding non-video block In some cases, two or more video blocksmay share a same non-video block.

In an example that the non-video content comprises audio content, avideo frame may include a scene having a singing bird 1401 (representedby 2 video blocks) and a silent stone 1402 (represented by 16 videoblocks) which are far away from each other (meaning that the sound thatthe singing bird 1401 makes may not be heard at the location where thesilent stone 1402 resides), video blocks associated with the singingbird 1401 may have corresponding audio blocks, and video blocksassociated with the silent stone 1402 might not have corresponding audioblocks. When a user focuses on the singing bird 1401, the video blocksrepresenting the singing bird 1401 may be transmitted to the user devicein a first bitrate, and the corresponding audio blocks may betransmitted to the user device with a second bitrate, such as a secondbitrate selected by the user. When the focus of the user shifts to ablock (e.g., on block B_(8,5)) located between the singing bird 1401 andthe silent stone 1402, the audio blocks corresponding to the singingbird 1401 may be transmitted at a bitrate version lower than the secondbitrate selected by the user since the user expects fading sound fromthe singing bird 1401. When a user focuses on the silent stone 1402, thevideo blocks representing the silent stone 1402 may be transmitted tothe user device, and corresponding audio blocks might not be transmittedto the user device.

In another example, a video frame includes a scene having a singlecharacter playing symphonic orchestra music. Since a consumer of suchscene is generally more interested in audio content than the videocontent, when the consumer selects the music playing scene for display,the video blocks representing the speech scene may be encoded fortransmission using a lower bitrate within a bitrate range with whichvideo content is normally encoded, and the corresponding audio blocksmay be encoded for transmission using a higher bitrate within a bitraterange with which audio content is normally encoded. In some examples,information of scene may be retrieved from the metadata of the videocontent. In this way, network bandwidth may be saved without negativelyimpacting the consumer satisfaction, and the server may dynamicallyadjust bandwidth allocation between the video content and audio contentthat associated with the video content.

In some examples, the non-video content may comprise closed captioning(CC) content.

CC content may, for example, contain textual transcripts of spoken wordsin an audio track that accompanies the video stream. In some cases whereCC content is embedded in video content, when a user moves his eyes, theCC may move along with the user's eye movement so that the CC text mayalways, for example, appear at the lower-right of the user's field ofview. In other cases where CC content is not embedded (non-embedded CC)in the video content, when a user moves his eyes, the non-embedded CCdoes not move corresponding to the user's eye movement, that is the CCtext may always, for example, appear at the center of the lower portionof the video frame. In a video frame, the region covered by the CCcontent may be encoded for transmission twice along with two types ofblocks, one with CC content and one without CC content, and the systemmay transmit and/or decode the correct version depending on whether CCcontent is to appear. Examples of transmission of a video frameincluding CC will be described in details in FIGS. 15 and 16.

FIG. 15 illustrates an example of delivering embedded closed captioningcontent associated with a video frame 150 according to one or moreillustrative aspects of the disclosure. In this example, a regioncovered by CC content (e.g., textual transcripts “Live From ComcastSports”) may correspond to two sets of blocks B_(i,j) (e.g., B_(9,2),B_(10,2), B_(11,2), and B_(12,2)) which are blocks without CC content,and B″_(i,j) (e.g., B″_(9,2), B″_(10,2), B″_(11,2), and B″_(12,2)) whichare blocks embedded with CC content. When a user selects to turn off CCcontent, the server may deliver blocks B_(9,2), B_(10,2), B_(11,2), andB_(12,2) which do not include CC content. When a user selects to turn onCC content, the server may deliver blocks B″_(9,2), B″_(10,2),B″_(11,2), and B″_(12,2) which include CC content. Block B_(i,j) may beencoded for transmission using bitrate R_(i,j) which may a same ordifferent bitrate as bitrate R″_(i,j) of block B″_(i,j). In some cases,blocks B″_(9,2), B″_(10,2), B″_(11,2), and B″_(12,2) covered by CCcontent may be encoded for transmission using a bitrate lower than thebitrate encoded with surrounding blocks not covered by CC content.

FIG. 16 illustrates an example of delivering non-embedded CC contentassociated with a video frame 160 according to one or more illustrativeaspects of the disclosure. In this example, CC content is not embeddedin video content but covers video content in some region of a videoframe. As shown in FIG. 16, the region covered by CC content (e.g.,textual transcripts “Live From Comcast Sports” with a black background)may correspond to two sets of blocks B_(i,j) (e.g., B_(9,2), B_(10,2),B_(11,2), and B_(12,2)) which are video blocks without CC content andblocks B′″_(i,j) (e.g., B′″_(9,2), B′″_(10,2), B′″_(11,2), andB′″_(12,2)) which are CC content blocks including CC content with ablack background. In this case, when a user selects to watch CC content,the server delivers B′″_(9,2), B′″_(10,2), B′″_(11,2), and B′″_(12,2) tothe user device. Alternatively, the region covered by CC content (e.g.,textual transcripts “Live from Comcast sports” with a black background)may correspond to one set of blocks B′″_(i,j) (e.g., B′″_(9,2),B′″_(10,2), B′″_(11,2), and B′″_(12,2)) which are CC content blocksincluding CC content with a black background. When a user selects towatch CC content, the server delivers B′″_(9,2), B′″_(10,2), B′″_(11,2),and B′″_(12,2) to the user device. The user device may download the CCcontent blocks and might not download the video content covered by theCC content blocks. It should be understood that CC content may be lockedto a fixed location for each video frame, or alternatively, may movearound among different video frames.

In the aforementioned examples, delivery of video content via acommunication system may be applicable to delivery of 2D video content.The present discourse is not limited thereto, and may be applied todelivery of three-dimensional (3D) video content. 3D video content, suchas pre-recorded or live 3D video content, may be offered by one or more3D content sources. The 3D content sources may capture video 3D contentusing one or more cameras. Cameras may be any of a number of camerasthat are configured to capture video content. Cameras may be configuredto capture two off-set two-dimensional (2D) video content for a left eyeand a right eye, respectively, of a user. The captured 2D video contentmay be used for generation of 3D video content for transmission to auser device. The user device may be a television display device and/orsome other computer implemented device where different video content maybe supplied to eyes of a user by two different outputs.

FIG. 17 illustrates an example of delivering 3D video content accordingto one or more illustrative aspects of the disclosure. In this example,in order to provide a stereoscopic view to a user, the server (at leastone of the content server 306 and the application server 307) mayprovide to the user device 1766, one encoded 2D video frame 1702 for theright eye of the user and another encoded 2D video frame 1704 for theleft eye of the user. The encoded 2D video frames 1702 and 1704 mayinclude an identical scene but are captured and tailored for right andleft eyes respectively, and may be delivered to the user device 1766simultaneously or alternately. The server may package and encode thevideo frame 1702 independently of the video frame 1704. The server mayencode each block of the video frame 1702 for transmission using arespective bitrate Rr_(i,j) and may encode each block of the video frame1704 for transmission using a respective bitrate Rl_(i,j). The bitrateRr_(i,j) and the bitrate Rl_(i,j) may be a same or a different bitrate.The user device 1766 may receive and decode the two video frames 1702and 1704 for display. Through a pair of 3D eyeglasses 1710, the left eyeof the user sees the decoded 2D video frame 1704 for the left eye, andthe right eye of the user sees the decoded 2D video frame 1702 for theright eye.

In some cases where one eye (e.g., the right eye) of a user may havebetter vision than the other (e.g., the left eye), the server may encodeblocks of the video frame 1702 for transmission using a higher bitrateand encode blocks of the video frame 1704 for transmission using a lowerbitrate. In some examples, during a training procedure, the user mayinput his vision profile into the user device, and the user device 1766may transmit the user's vision profile to the server. The vision profilemay include visual acuity, refractive error, hyperopia, myopia,astigmatism, and etc. The server may receive and register the user'svision profile in a database. Based on the user's vision profile, theserver may apply higher bitrates to video frames for the eye with bettervision and apply lower bitrates to video frames for the eye with worsevision. Alternatively, the server may apply a same bitrate to both videoframes and apply enhancement layers for the eye with better vision.

In some cases, the field of view and area of focus (or the aggregatearea of focus) of the left eye may be substantially identical to thoseof the right eye. Methods of delivery of a video frame described inFIGS. 3A through 16 may be utilized to deliver each of video frames 1702and 1704.

In some examples, to increase comfortability of a user watching astereoscopic/3D video, the CC content for the stereoscopic/3D video maybe placed to the user at a perceived distance which is substantiallyequal to a perceived distance from the main video content (e.g., thevideo content within the area of focus) to the user. This can be done bysynchronizing metadata on the perceived depth of the CC content andmetadata on the perceived depth of the main video content. The metadataon perceived depth of video content may be obtained from one or more ofdepth captures as part of the camera rigs, computed using videogrammetry techniques, or editor-provided minimum-distance figures.

In some examples, the server may provide both the 2D and 3D versions ofvideo content to satisfy both 2D and 3D display devices. In theseexamples, when the user makes a sudden movement like turning his head orsitting down, a 3D effect may be disorienting, and in those situation,the server may temporarily send only the 2D version of the video contentinstead of the 3D version, to help avoid that disorientation. Forexample, when a user is viewing a 3D video with a 3D user device, the 3Ddisplay device tracks movement of eyes of the user and transmitsinformation representing the eye movement to the server. The serverreceives the information and determines that 3D image can be renderedwith such movement, the server may compensate for such movement orrotation by modifying the encoded video frames for at least one eye.When the server determines that the 3D image cannot be rendered withsuch movement, the server may begin delivering the 2D version of thevideo content to the user device. In some examples, the server mayswitch back to 3D image from the 2D version after a certain time period.To realize the switching from 3D image to 2D image, the server mayeither switch from stereoscopic video to monoscopic video by showing asame video to both eyes, or only show the video to one of the eyes andblock the video to the other eye. In the case that a user initially sitson a sofa and suddenly lies down, the server may consider that theamplitude of such movement exceeds the threshold and indicates that the3D image cannot be rendered. It should be understood that the abovemodifying may be triggered in some cases regardless of user movement,e.g., when monitoring tools/analysis indicate that the depth image isundesirable.

FIG. 18 is an exemplary flow diagram of a method for transmitting videocontent according to one or more illustrative aspects of the disclosure.It should be understood that the steps in FIG. 18 are illustrated by wayof example and do not limit the scope of the present disclosure.

The process starts from step S1802 in which the server receives a newvideo file including a plurality of video frames.

At step S1804, each of the plurality of the video frames may bepartitioned into a plurality of blocks (e.g., FIGS. 3A and 3B).

At step S1806, since the requested video frames have never beenpreviously requested and transmitted to any user devices, the server maydetermine initial configuration, e.g., bitrate, of each of the blocks ofeach video frame. The server may predetermine or receive informationrepresenting one or more bitrate versions (e.g., a lowest possiblebitrate to a highest possible bitrate and certain bitrate points inbetween) for each block based on metadata included in the video file andprovided by the video editor.

At step S1808, one or more user devices send request(s) to the serverfor accessing content. The content may include video content andoptionally non-video content (e.g. FIG. 14), e.g., associated audiocontent. In this step, the server may determine whether the user devicehas the authorization to access the requested content based on certaincriteria, e.g., the service subscription of the user device. Upon thedetermination that the user device has the authorization to access therequested video content, the process starts from the first video frameof the plurality of the video frames and proceeds to the next stepS1810. Upon the determination that the user device does not haveauthorization to access the requested video content, the processproceeds to the end S1880.

At step S1810, the server may determine whether a report, whichindicates, for example, what user devices were looking at in therequested video frame, has been received from the user devices. For thecontent that has been previously accessed, the report may be transmittedfrom the one or more user devices that have previously accessed thevideo frame. It is noted that for the first-time requested content, theserver has not received any reports from user devices, and in this case,the process proceeds to the next step S1820.

When the server determines that a report is received by the server, theserver may proceed to steps S1812-S1818. At step S1812, the server mayuse data included in the report to identify the sender (e.g., the one ormore user devices) and a type of the sender; at step S1814, the servermay identify the field of view of the user of the sender and the area offocus of the user of the sender; and at step S1816, the server mayreceive statistical data comprising user's feedback onpreviously-delivered video frame(s). At step S1818, the server may storethe data included in the report in a database for determining whichblocks of the video frame to be sent for future requests.

At step S1820, the server may determine, based on certain criteria,e.g., a field of view that determined at step S1814, whether there areany blocks of the video frame to be transmitted to the user device. Whenthe field of view is determined by the video editor, especially for thefirst-time requested content, metadata may be used to carry informationrepresenting a predetermined field of view. When the server determinesthat there are at least some of the blocks of the video frame are to betransmitted to the user device, the process proceeds to step S1812. Ifthe server determines that there are no blocks of video frame are to betransmitted to the user device, the process proceeds to step S1866. Forexample, in a video game, when a game character of a user has noequipment to see anything in a blackout room (a scene shown by the videoframe), the server may determine that no content portion of the videoframe should be sent to the user device.

At step S1822, the server may check the access request, which includesdata representing a user device ID, from the user device to identify theuser device who requests the content.

At step S1824, the server may receive and retrieve a vision profile ofthe user of the user device from the user device. As discussed above,when the user's vision profile shows that the user's vision exceeds apredetermined threshold, the server may unicast an additionalenhancement layer applied to the area of focus of the user to the userdevice. By doing so, the resolution of the user's area of focus may beincreased. When the user's vision profile shows that the user's visionis below a predetermined threshold, the server may determine not totransmit blocks representing video content that might not be seen by theuser, because of his lower vision profile.

At step S1826, as discussed above, when the server determines that therequested content includes non-video content, e.g., audio content,associated with the video content, the server may retrieve information(e.g., metadata) of the audio and video content for dynamicallyadjusting, for transmission, bandwidth allocation between the videocontent and the audio content (details have been discussed in thedescription of FIG. 14). Alternatively or in addition, as discussedabove, the server may retrieve metadata provided by the video editor fordynamically adjusting, for transmission, field of view and area of focus(e.g., FIG. 9). In some embodiments, step S1826 may be affected by stepS1824 since when the user's vision allows the server not to transmitcertain video blocks to the user, the server may allocate more bandwidthto the audio content.

At step S1830, the server may determine whether the CC is turned on bythe user device.

When the server determines that the CC is turned on by the user device,the process proceeds to step S1832. At step S1832, when the CC is turnedon and the CC content is not embedded in video content (as shown in FIG.16), the server may identify video blocks that are covered by CCcontent. As such, the server may determine to omit these identifiedvideo blocks in step S1850 which will be described later. In other caseswhich are not shown in FIG. 18, when the CC is embedded in video contentas shown in FIG. 15, the server may identify blocks B″_(9,2), B″_(10,2),B″_(11,2), and B″_(12,2) which include CC content.

As previously discussed, if the user watching 3D content and makes asudden movement, the server may temporarily disable the 3D effect tominimize disorientation. In step 1840, the server may determine whetherit has received an indication that the user has made such a movement, orthat a 3D effect should be temporarily disabled, and if so, then in step1842, the server may store information indicating that this is to bedone for the subsequent frame. When the server receives data indicatingthat such movement is complete, the server may re-enable the 3D effect.

At step S1850, the server may determine blocks that can be omittedduring the transmission of the video frame. As discussed above, for a360 degree video frame, when a user is wearing a VR headset, the fieldof view may change depending on the viewing direction of the user. Theserver may determine blocks outside of the field of view are not to bedelivered to the user device. In some examples, the blocks that are notto be delivered to the user device may be blocks that are disposed 180degrees the opposite of where the user is looking (e.g., directly behindthe user or an opposite focal point), and blocks surrounding thatopposite point. In addition, as discussed in step S1832, the blocksobscured by non-embedded CC (as shown in FIG. 16) may be omitted duringthe transmission.

At step S1852, if multiple user devices request the video content, basedon the retrieved data indicating multiple areas of focus of the users ofthe multiple user devices, the server may determine an aggregate area offocus for the users of the multiple user devices as discussed above. Atstep S1854, as discussed above, the server may determine a subset ofblocks disposed within or overlapping the aggregate area of focus (e.g.,FIG. 13).

At step S1856, when multiple user devices request the video frame, theserver may determine blocks of the video frame to be included inbaseline multicast streams at a baseline resolution for multicasting toall user devices. The blocks to be multicast to all user devices mayinclude all blocks of the video frame, or alternatively, blocks disposedwithin or overlapping all the fields of view of the multiple userdevices and optionally additional blocks surrounding the fields of viewof the multiple user devices (e.g., FIG. 10).

At step S1858, the server may determine the blocks to be applied withenhancement layer(s) for an increased resolution. In some examples wheremultiple user devices request the video frame, the server may determineto apply an enhancement layer, for multicasting to all user devices, toblocks disposed within or overlapping the aggregate area of focus. Theserver may determine to apply a respective enhancement layer, forunicasting to a respective user device, to additional blocks disposedwithin or overlapping a respective area of focus of a user of therespective user device. In some other examples where a single userdevice is requesting the video frame, the server may determine to applyan enhancement layer, for unicasting to the single user device, toblocks disposed within or overlapping the area of focus of the singleuser .

At step S1860, based on results of steps S1856 and S1858, the server maycreate multicast streams at the baseline resolution and createenhancement layers for the targeted blocks.

At step S1862, the server may multicast the baseline multicast streamsto the user devices identified in step S1822. The server may multicastenhancement layers created in step S1860 to user device(s) requestingthe content.

As noted above, in some cases where the user device is reported to be aDVR in step S1812 and the server receives data indicating that the DVRis recording the content while no one is watching, the server may inferthe aggregate area of focus determined based on all of the areas offocus of the other users, determine which block(s) to be omitted fortransmission based on the aggregate area of focus, determine subset ofblocks for aggregate area of focus, determine content for the baselinemulticast stream and content for enhancement multicast stream based onthe aggregate area of focus, create the multicast streams, and deliverthe video frame along with an enhancement layer applied to the aggregatearea. In these cases, step S1852 is performed before step S1852 mayswitch order.

At step S1864, as discussed above, the server may update the bitrateversions for each block by deleting certain bitrate versions based onthe statistical data transmitted from the user devices at step S1816(e.g., FIG. 13).

At step S1866, as discussed in FIG. 13, when the statistical dataindicates that a bitrate version of a block has received a percentage ofusers' attention that is lower than a minimum required percentage, theserver may determine that this bitrate version of the block may bedeleted from the storage of the server and be moved to and stored in astorage of a higher level of server. In some examples, the server maydetermine that a block itself may be deleted from the storage of theserver and be moved to and stored in a storage of a higher level ofserver when the statistical data indicates that this block has not beenrequested or transmitted to any user devices.

At step S1868, the server may delete the bitrate version of a block or ablock itself based on the determinations of step S1866. As noted above,however, these blocks may be retrieved from a higher level server if theblocks are by a user in the future.

At step S1870, the server determines whether the video frame that hasbeen transmitted to the requested user device(s) is the last frame inthe request content. If the video frame is the last frame, the processends at S1880. Otherwise, the process goes back to step S1810 forprocessing the next video frame.

It should be understood that the steps in the flow diagram illustratedin FIG. 18 need not all be performed in the order specified and somesteps may be omitted, changed in order, or performed simultaneously. Inone example, steps S1812 to S1818 may be performed in different orders.In another example, step S1840 may be omitted if only 3D version of thevideo content is provided to the user devices. In another example, thepartitioning may be done in a real-time, e.g., partitioning a videoframe upon the video frame being requested for access, and thus the stepS1804 may be performed after step S1808.

It should also be understood that the description of the aforementionedfigures can detail some steps shown in FIG. 18.

Additional alterations, modifications, and improvements as are madeobvious by this disclosure are intended to be part of this descriptionthough not expressly stated herein, and are intended to be within thespirit and scope of the disclosure. Accordingly, the foregoingdescription is by way of example only, and not limiting. This patent islimited only as defined in the following claims and equivalents thereto.

1. A system comprising: a computing device; and a first user deviceassociated with a first user, wherein the computing device comprises:one or more processors; and memory storing executable instructions that,when executed by the one or more processors of the computing device,cause the computing device to: determine, from a partitioned video framecomprising a plurality of blocks, a common area of focus that is commonto a first area of focus associated with the first user and a secondarea of focus associated with a second user; send, to the first userdevice associated with the first user and to a second user deviceassociated with the second user, a baseline layer comprising blocksoverlapping the common area of focus and blocks outside of the commonarea of focus, wherein in the baseline layer, the blocks overlapping thecommon area of focus have higher resolution than the blocks outside ofthe common area of focus; and send, to the first user device, anenhancement layer corresponding to blocks that are outside of the commonarea of focus and overlapping the first area of focus; and wherein thefirst user device comprises: one or more processors; and memory storingexecutable instructions that, when executed by the one or moreprocessors of the first user device, cause the first user device to:receive the baseline layer and the enhancement layer.
 2. The system ofclaim 1, wherein the executable instructions in the memory of thecomputing device, when executed by the one or more processors of thecomputing device, cause the computing device to: send the baseline layervia a multicast transmission; and. send the enhancement layer via aunicast transmission.
 3. The system of claim 1, wherein the executableinstructions in the memory of the computing device, when executed by theone or more processors of the computing device, cause the computingdevice to: send, to the second user device, a second enhancement layercorresponding to blocks that are outside of the common area of focus andoverlapping the second area of focus.
 4. The system of claim 1, whereinthe executable instructions in the memory of the computing device, whenexecuted by the one or more processors of the computing device, causethe computing device to: determine a portion, of the common area offocus, that receives more attention, from the first user, the seconduser, and other users, than other portions of the common area of focus.5. The system of claim 1, wherein the executable instructions in thememory of the computing device, when executed by the one or moreprocessors of the computing device, cause the computing device to:determine, based on a resolution of a block of the plurality of blocks,a bitrate at which audio content associated with the block is sent. 6.The system of claim 1, wherein the executable instructions in the memoryof the computing device, when executed by the one or more processors ofthe computing device, cause the computing device to: record, based on aresolution of the blocks overlapping the common area of focus, thepartitioned video frame.
 7. The system of claim 1, wherein theexecutable instructions in the memory of the computing device, whenexecuted by the one or more processors of the computing device, causethe computing device to: receive, from the first user device, a visionprofile of the first user; and based on the vision profile of the firstuser, send a first two-dimensional video frame for one eye of the firstuser and a second two-dimensional video frame for the other eye of thefirst user, wherein a resolution of the first two-dimensional videoframe is higher than a resolution of the second two-dimensional videoframe.
 8. A method comprising: determining, by a computing device andfrom a partitioned video frame comprising a plurality of blocks, acommon area of focus that is common to: a first area of focus associatedwith a first user, and a second area of focus associated with a seconduser; sending, to a first user device associated with the first user andto a second user device associated with the second user, a baselinelayer comprising blocks overlapping the common area of focus and blocksoutside of the common area of focus, wherein in the baseline layer, theblocks overlapping the common area of focus have higher resolution thanthe blocks outside of the common area of focus; and sending, to thefirst user device, an enhancement layer corresponding to blocks that areoutside of the common area of focus and overlapping the first area offocus.
 9. The method of claim 8, wherein the sending the baseline layercomprises sending the baseline layer via a multicast transmission; and.wherein sending the enhancement layer comprises sending the enhancementlayer via a unicast transmission.
 10. The method of claim 8, furthercomprising: sending, to the second user device, a second enhancementlayer corresponding to blocks that are outside of the common area offocus and overlapping the second area of focus.
 11. The method of claim8, further comprising: determining a portion, of the common area offocus, that receives more attention, from the first user, the seconduser, and other users, than other portions of the common area of focus.12. The method of claim 8, further comprising: determining, based on aresolution of a block of the plurality of blocks, a bitrate at whichaudio content associated with the block is sent.
 13. The method of claim8, further comprising: recording, based on a resolution of the blocksoverlapping the common area of focus, the partitioned video frame. 14.The method of claim 8, further comprising: receiving, from the firstuser device, a vision profile of the first user; and based on the visionprofile of the first user, sending a first two-dimensional video framefor one eye of the first user and a second two-dimensional video framefor the other eye of the first user, wherein a resolution of the firsttwo-dimensional video frame is higher than a resolution of the secondtwo-dimensional video frame.
 15. A non-transitory computer readablemedium storing instructions that, when executed, cause: determining, bya computing device and from a partitioned video frame comprising aplurality of blocks, a common area of focus that is common to: a firstarea of focus associated with a first user, and a second area of focusassociated with a second user; sending, to a first user deviceassociated with the first user and to a second user device associatedwith the second user, a baseline layer comprising blocks overlapping thecommon area of focus and blocks outside of the common area of focus,wherein in the baseline layer, the blocks overlapping the common area offocus have higher resolution than the blocks outside of the common areaof focus; and sending, to the first user device, an enhancement layercorresponding to blocks that are outside of the common area of focus andoverlapping the first area of focus.
 16. The non-transitory computerreadable medium of claim 15, wherein the instructions, when executed,cause: sending the baseline layer via a multicast transmission; and.sending the enhancement layer via a unicast transmission.
 17. Thenon-transitory computer readable medium of claim 15, wherein theinstructions, when executed, cause: sending, to the second user device,a second enhancement layer corresponding to blocks that are outside ofthe common area of focus and overlapping the second area of focus. 18.The non-transitory computer readable medium of claim 15, wherein theinstructions, when executed, cause: determining, based on a resolutionof a block of the plurality of blocks, a bitrate at which audio contentassociated with the block is sent.
 19. The non-transitory computerreadable medium of claim 15, wherein the instructions, when executed,cause: recording, based on a resolution of the blocks overlapping thecommon area of focus, the partitioned video frame.
 20. Thenon-transitory computer readable medium of claim 15, wherein theinstructions, when executed, cause: receiving, from the first userdevice, a vision profile of the first user; and based on the visionprofile of the first user, sending a first two-dimensional video framefor one eye of the first user and a second two-dimensional video framefor the other eye of the first user, wherein a resolution of the firsttwo-dimensional video frame is higher than a resolution of the secondtwo-dimensional video frame.